This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-07-11
Channels
- # announcements (1)
- # aws (3)
- # beginners (48)
- # calva (2)
- # cider (47)
- # clj-kondo (1)
- # cljs-dev (23)
- # cljsrn (10)
- # clojure (81)
- # clojure-chicago (4)
- # clojure-europe (3)
- # clojure-greece (4)
- # clojure-italy (8)
- # clojure-losangeles (1)
- # clojure-nl (6)
- # clojure-sanfrancisco (1)
- # clojure-seattle (1)
- # clojure-uk (21)
- # clojurescript (40)
- # core-async (82)
- # cursive (18)
- # datomic (6)
- # duct (11)
- # figwheel-main (4)
- # fulcro (26)
- # jobs-discuss (22)
- # leiningen (18)
- # off-topic (10)
- # pathom (3)
- # re-frame (5)
- # reagent (16)
- # reitit (4)
- # shadow-cljs (8)
- # specter (7)
- # sql (16)
- # tools-deps (58)
- # xtdb (30)
is there any way of loading clojure code on runtime that is outside of the classpath? I guess this is security issue, but I was hoping to add some preloaded code to my application in similar fashion to the #'user namespace.
Calling load-file
can do it, as well as eval
I'll try further, but I'm trying to call functions from the classpath namespaces. And from load-file, I can't call the symbols defined in the file. Maybe I'm too quick asking for help.
By classpath namespaces, you mean Clojure namespaces whose source files are on the classpath?
Macros are called and executed during eval of the data structures representing the code, so not sure whether that will prevent you from doing what you wish.
You are calling load-file
on a file containing Clojure source code, and you say you "can't call the symbols defined in the file" -- from where can't you call the symbols defined in the file?
yes, this is a file I'm trying to load outside of the classpath
(ns panaeolus.csound.instruments.spacedrone
(:require [panaeolus.csound.macros :as c]))
(c/definst spacedrone
:orc-internal-filepath
"instruments/drone/spacedrone.orc"
:instr-form [{:name :dur :default 4}
{:name :nn :default 48}
... etc....
`
(load-file "/full/path/to/spacedrone")
or read-string + eval
from the namespace I call load-file, I can't see the symbol spacedrone defined. But maybe there's error with the macro not related to load-file.from which namespace do you call load-file?
Have you tried removing most of the code in namespace p.c.i.spacedrone, leaving only say one defn of a simple function, and see if that exists after doing load-file
?
just for testing purposes, to see if it works as you hope, maybe try adding an ns
form to the file you are calling load-file
on.
yep. For me as well with a simple try from the REPL
Is there a place one can find open-source Clojure projects that would like contributions?
http://open-source.braveclojure.com/ might be such a place
You can also have a look at the clj-commons github org: https://github.com/clj-commons
(-> x
(blah :y)
(cond-> (pred? x) (blop :z))
(bleep))
vs
(cond-> x
:always (blah :y)
(pred? x) (blop :z)
:always (bleep))
what do you think?plus you can keep adding functions like bleep
without needing :always
Interesting.
(ns private.tmp.import
(:import (java.util.regex.Pattern))) ;; not allowed by core.specs
(ns private.tmp.import
(:import (java.util regex.Pattern))) ;; allowed by core specs
spotted the latter here: https://github.com/clojure/tools.reader/blob/master/src/main/clojure/clojure/tools/reader.clj#L27
@U04V15CAJ maybe even more surprising is that you don't even need the import if you don't need the alias Pattern
and use it with package prefix.. I've learnt this recently after asking a similar question here in slack. Not sure if it has always been there
I think it's the same in Java: importing classes don't change anything to the classpath, it's just a syntactic convenience
this makes it a little harder on the linter: you don't know if something is a class or a namespace (that was or wasn't required)
I guess the linter isn't on the same classpath? if so, you could try to resolve
unless you have better options. A good heuristic would be camelcase vs non-camelcase <=> java vs non-java
looking for an example resource of how to lazily stream a resultset from next.jdbc
...
I came here with very "Clojure way of doing things" question.
Clojure is great for data pipelines. I would like to prove this to my coworkers for maybe one day, introduce Clojure has an option for some of our tasks.
But I miss some full pratical example.
The use case should be the following :
- Receive a data file (e.g. CSV)
- Parse it
- Validate datas
- Insert valid ones into a database, Write others into a file (with some explain-data)
What's come to mind could be some sort of "Railway" (with https://github.com/funcool/cats ?). But I'm not sure that's the more "Clojurey" solution.
From this talk (https://www.youtube.com/watch?v=oOON--g1PyU), I know halt-when
is good to stop a transduction operation.
Which is good for "one shot" loading like the one expose by Stuart Halloway, but not for my imaginary (but realistic) use case.
Do you have any recommended solution (or reading) ?
Before you dive into Cats, I recommend reading this blog about it: https://medium.com/@functionalhuman/functional-programing-with-cats-105d666b260e
Do note that the examples at the end utilize clojure.spec.alpha2 which is still in prerelease.
Just discover your answer. I'll have a look, thanks.
Hmm. In Clojure when you write:
(ns foo)
bar/baz
bar/baz
could in theory be a public field in a class bar
right? so it's hard to say this is an unresolved symbol by linting staticallyyes, w/o keeping a runtime mapping of the available classes it's not easy to distinguish the two
currently clj-kondo doesn't report anything with a namespace as unresolved, but thinking about this more, I think it's the correct behavior because of the above
@charles.fourdrignier You could maybe draw some inspiration from Grammarly's experience optimizing some data munging with transducers https://tech.grammarly.com/blog/building-etl-pipelines-with-clojure
Thanks ! To be honest, I already found it. It's very interesting for the performance point of view. But at the moment my biggest concern is about 1 entry -> 2 outputs, which seems not to be a so common pattern in Clojure. By looking with a fresh eye to the article, maybe introducing a second "out" channel should be a good idea ?
If data has lots of interdependencies, it maybe harder to parallelize or design transducers around it. But in general, I think forking data out to different places can be an orthogonal concern.
a solution to this problem that i use sometimes is to map over your data, returning command data structures (often just a pair vector) that will then be "interpreted"
sounds like a lot as i read it, but it's actually pretty simple, and you segregate IO in your "interpreter" function. Interpreter is a big word for just a case statement =)
Thank you for the ideas.
Yeah, I use the @U13RRNF2M pattern a fair bit - create a sequence of ops
like [{:op/type :write-to-error-log :op/data …} {:op/type :store-in-db :op/data …} ...]
so all the clever stuff happens in the generation of the ops sequence, and can check it works without doing any io
@jonpither If you're also interested in current.jdbc, I asked this question 3 years ago on SO: https://stackoverflow.com/questions/39765943/clojure-java-jdbc-lazy-query
@jonpither I would recommend next.jdbc/plan
(or clojure.java.jdbc/reducible-query
). There are several DB-specific concerns as well to get true streaming. On some DBs you need the connection to have auto-commit turned off and/or :fetch-size 0
I think.
@seancorfield do you happen to have an example of next.jdbc/plan being used to produce a lazy seq of results?
No, you can't produce a lazy sequence from it. Laziness cannot escape the scope of the connection. But "lazy seq" is not the same as "lazily streaming results from the DB".
You have to process the whole result set while the connection is still open. With plan
(and its forefather reducible-query
), you reduce the whole result set, but you can process more rows than will fit in memory by setting the appropriate (DB-specific) set of options. So exactly how you do it, will depend on your database.
thanks @seancorfield, will ponder
And if you follow-up in #sql I'm more likely to see the questions. I'd missed your original question here but happened to see Michiel's response.
ah ok, sorry, had looked for jdbc @seancorfield
how might i go about deploying a deps.edn
based project to a private s3 repo without using lein
?
... just kidding. https://juxt.pro/blog/posts/pack-maven.html 🙂
since where on deps.edn topic, is there a good set of practices for managing a monorepo with deps. Do monorepos with tools.deps?
There was a discussion on tools.deps and monorepos recently on the #tools-deps channel, perhaps yesterday. Should still be in the history on Slack, but if not, https://clojurians.zulipchat.com has it in its unlimited history.
Ah, I need to start using the search for channels