This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-02-02
Channels
- # announcements (3)
- # asami (29)
- # babashka (62)
- # beginners (131)
- # biff (7)
- # calva (31)
- # cider (5)
- # clerk (14)
- # clj-kondo (3)
- # cljsrn (12)
- # clojars (18)
- # clojure (72)
- # clojure-austin (17)
- # clojure-dev (6)
- # clojure-europe (31)
- # clojure-indonesia (1)
- # clojure-nl (1)
- # clojure-norway (18)
- # clojure-sweden (11)
- # clojure-uk (6)
- # clr (47)
- # conjure (42)
- # cursive (88)
- # datalevin (2)
- # datomic (25)
- # emacs (42)
- # exercism (1)
- # fulcro (10)
- # funcool (8)
- # gratitude (2)
- # honeysql (16)
- # introduce-yourself (5)
- # jobs-discuss (26)
- # leiningen (5)
- # lsp (31)
- # malli (21)
- # matcher-combinators (14)
- # missionary (2)
- # nbb (1)
- # off-topic (40)
- # pathom (38)
- # portal (2)
- # re-frame (7)
- # reagent (18)
- # reitit (1)
- # releases (5)
- # shadow-cljs (62)
- # sql (12)
- # testing (4)
- # xtdb (37)
How can I skip the fist N rows of a file using clojure.data.csv/read-csv
?
you can't directly, but you could arrange to have a reader/string that drops n lines
if the file is "short", you could slurp it, then split-lines, drop N, then string/join back to a string
its 500KB
seems easily slurp-able, if performance is not critical
a few extra ms is just fine
I think I got it:
(def raw-data (clojure.string/join (drop 4 (clojure.string/split-lines (slurp "foo.csv")))))
So my files are weird and are UTF-16 little endian. Slurp is not detecting the encoding, or at least pprint isnt printing it right
But it's not guaranteed to be UTF-16 LE — it might be UTF-8 or ISO-8859-1. Any way to detect the encoding to pass to slurp?
you can pass an encoding to slurp
(slurp "foo.csv" :encoding "UTF-16")
I think
if you don't know the encoding, not sure what exists in the JDK to do that detection (wouldn't surprise me if there is something, but don't know)
I found [clj-det-enc "1.0.0"]
but it operates on a stream and I have a string at this point
so i created this (defn detect-encoding [stream] (det/detect stream))
but i dont know how to pass a string to it (sorry i'm on week 2 with Clojure)
http://clojure.java.io/input-stream can give you an input stream from a file
okay so i have to read the file twice - once to detect the encoding, and then again with slurp?
yeah, I assume detect is not actually going to read the whole file?
yeah my guess is if it's written well it only reads the first few bytes
ok so i have this:
(defn detect-encoding [file] ((det/detect ( file))))
but I get an exception here:
(p/pprint (detect-encoding "foo.csv"))
> Execution error (ClassCastException) at whatsnew-clj.core/detect-encoding (core.clj:14).
> class java.lang.String cannot be cast to class clojure.lang.IFn (java.lang.String is in module java.base of loader 'bootstrap'; clojure.lang.IFn is in unnamed module of loader 'app')seems like you have an extra set of parens in there?
just in general, you should like questioningly any time you have more than 1 ( stacked up
dont think so — i mean it compiles 😉
ah i see now!
it prints "UTF-16LE"
so i know it works 😉
ok very nice now:
(def filename "foo.csv")
(def encoding (detect-encoding filename))
;; drop the first 4 lines of the file (we don't need the header info)
(def raw-data
(clojure.string/join
(drop 4
(clojure.string/split-lines
(slurp filename :encoding encoding)))))
(defn -main
[& args]
(p/pprint raw-data))
Does anyone know of a clojure library that can generate api client code from an openapi spec? I tried the openapi-generator but it's not very nice to try to tweak the output, and in a large spec it's a bit of a nightmare.
Martian is pretty amazing, I was able to use it to integrate 4 different apis into a new app in record time. Thanks again for the recommendation.
hi @alexmiller,
I checked this https://clojure.atlassian.net/browse/TLOG-28 and it seems that all that it's needed is to have the new slf4j 2.0 API on the classpath.
clojure tools-logging uses factory# (org.slf4j.LoggerFactory/getILoggerFactory)
which was migrated to use ServiceLoader: https://github.com/qos-ch/slf4j/blob/a5540ad51066b4b15132fdf27ead630519541d35/slf4j-api/src/main/java/org/slf4j/LoggerFactory.java#L107
(I don't have access to comment on the jira issue and I don't know a tools-logging specific channel ).
IMO nothing should be done for tools-logging to work with slf4j 2.0.
Clients just need to have slf4j 2.0 dependencies on the classpath.
I do recommend using 2.0 for dev profile as 1.7x will not be maintained.
you can comment on the Ask Clojure question linked at the top: https://ask.clojure.org/index.php/12390/provide-slf4j-2-0-factory-for-tools-logging
tools.logging does pull in slf4j 1.7.32 as a test dependency right now, maybe that would need to be changed?
we don't use PRs on contrib projects, but I can put it in my list for tomorrow
I went ahead and just did that now
not going to release as, no change really
thanks for the info!
Is it possible to redef a variable “by reference”? Something to the effect of:
(def foo 1) (let [x (var foo)] (with-redefs [@x 3] foo)) --> 3
user=> (def foo 1) (let [x #'foo] (with-redefs-fn {x 3} (fn [] foo)))
#'user/foo
3
user=>
just to enable testing, seems like using protocols as the alternative can get cumbersome quick.
with-redefs captures the value of the var then sets the var to a new value, then runs your code, then in a finally sets the value of the var back to the old value
and that kind of mutating is problematic when combined with things like multithreading and laziness
https://www.reddit.com/r/Clojure/comments/ble9k4/dont_use_a_protocol_to_make_testing_easier/ This is interesting discussion.
not really, I am pretty confident in my opinion on this, having worked on a large clojure code base before with-redefs was added to clojure.core, where we had written our own version of the macro
back in the before time, when all vars where dynamically bindable, people would use that for testing
What I am wanting to do at a high-level is be able to refer to individual vars within my function, but I want to embellish those functions before they are used. I can pass them all in (“embellished”) but that seems tedious to “get a handle” to each. I could pass each one indiv… I don’t know, I just didn’t like where that was headed.
I trend toward agreeing with you though honestly, just thought coding was going to get too tedious without it.
I’m sort of exploring so plan on trying at least a couple of tacts. multi-method might work well.
they might, my big issue with multimethods is extending them is also global, so you can't just create a little stub inside a particular test, like you can using reify for protocols
if you haven't experimented with metadata extended protocols, I would check those out, I find that they get rid of a lot of what people dislike about using protocols
big plus 1 on the protocols. at metabase we’re very multimethod heavy. i made a protocol that would delegate to the multimethod during regular operation and allows me to pass in easy implementations in testing. huge fan.
Previously you’d have to write things like
(defmethod ddl/persist-table ::my-fake-driver-that-throws …)
and instead can now just spin up whatever reify IPersist
that you need as values in the test.
my example:
• protocol and an implementation that dispatches to a multimethod based on the db driver type (multimethods are ddl.i/unpersist!
and ddl.i/persist!
◦ https://github.com/metabase/metabase/blob/master/src/metabase/task/persist_refresh.clj#L49-L66
• tests which assert methods are called, throws when persisting the second table, persist! isn’t called, only unpersist!, etc
◦ https://github.com/metabase/metabase/blob/master/test/metabase/task/persist_refresh_test.clj#L133
To add a little more color, I would say that the “never use with-redefs” viewpoint is definitely not a consensus in the Clojure community. I’m not saying this to say that the “never with-redefs” people don’t have strong points, but I’ve seen it go badly both ways. Personally I tend to try to avoid with-redefs when at all possible by using DI, but occasionally the overhead isn’t worth it. Multithreaded concerns aren’t usually a big deal in tests since Kaocha doesn’t support multithreaded test runs anyways
I'm a big with-redef fan, have had 0 issues with them for over 4 years. But our code bases are made up of many micro-services, so none of them is like some enormous monolith monster, and also we never parallelized our tests.
That said, mocking out a global var seems a bit weird to me. Normally I use with-redef to mock a function. If I'm testing a function that calls another which does IO, I'll with-redef the function that does IO. And in general, I inject all dependencies, and wouldn't access them through a global. So maybe that reduces my blast radius for with-redef.
Could somebody explain to me what exactly is Fulcro library/framework? Is it like Django / Spring / Rails / Next.js ? Clojure on backend + frontend? in the past I've dabbled with Reagent for front-end and saw a lot of options for back-end, but it was always standalone libraries that gave you a lot of freedom to craft your own thing. I've been wondering how polished of a stack can be made in Clojure that can really do a lot of things out of the box or with some minor adjustments. In particular, I'm a big fan of state machines for UI development and noticed that Fulcro supports that kind of thing. Would love to hear what you guys think of it if you've used it! :hugging_face:
If no one can help here, there is a #C68M60S4F channel...
The Clojure Way ™️ is composition of libraries rather than frameworks...
Thanks @U04V70XH6 ❤️
Fulcro is still a library, it has some batteries included for a lot of things, but it's not quite a fully fledged framework like Django or rails. Fulcro rad is a step in that direction but you are still left with a lot of choices and things to decided and make on your own. If you have some time to grasp the concepts of Fulcro I'd stay it's really worth it. I personally started a Fulcro/pathom3 project and really appreciate the experience. I'd say it's harder to get going than reagent/reframe, because it does much more and there's less doc/examples in the wild, but it also solve more problems.