This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-12-11
Channels
- # adventofcode (31)
- # announcements (6)
- # babashka (5)
- # beginners (93)
- # calva (15)
- # cider (20)
- # cljs-dev (16)
- # clojure (159)
- # clojure-berlin (1)
- # clojure-dev (5)
- # clojure-europe (9)
- # clojure-italy (9)
- # clojure-losangeles (2)
- # clojure-nl (26)
- # clojure-spec (7)
- # clojure-uk (33)
- # clojurescript (54)
- # clojutre (5)
- # cursive (20)
- # datomic (23)
- # emacs (19)
- # events (1)
- # expound (1)
- # figwheel-main (1)
- # fulcro (104)
- # hyperfiddle (1)
- # jobs (1)
- # luminus (10)
- # malli (59)
- # nrepl (1)
- # off-topic (11)
- # pathom (5)
- # planck (15)
- # reagent (13)
- # reitit (8)
- # rewrite-clj (10)
- # ring-swagger (3)
- # shadow-cljs (129)
- # tools-deps (46)
- # xtdb (14)
- # yada (1)
Does anybody have any thoughts about relative merits of core.async, manifold, and promesa for asynchronous code?
I'm just learning about the topic and I'm wondering what the tradeoffs between the three are
Hi Team, I was trying to setup the Clojure code analysis with Leiningen as build tool and Sonarqube as code analysis tool. I am getting the below error while executing the stage
10:53:36 05:23:35.933 ERROR: Error during SonarQube Scanner execution
10:53:36 org.sonar.api.utils.command.CommandException: .IOException: Cannot run program "lein": error=2, No such file or directory
10:53:36 at org.sonar.api.utils.command.CommandExecutor.execute(CommandExecutor.java:102)
10:53:36 at org.sonar.plugins.clojure.sensors.CommandRunner.run(CommandRunner.java:36)
10:53:36 at org.sonar.plugins.clojure.sensors.CommandRunner.run(CommandRunner.java:50)
We are executing this from Jenkinsfile
And I am able to print the Leiningen version inside the workspace
10:53:19 + ./lein -v
10:53:19 which: no lein in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/alloy-jdk8-3.4.2/bin:/opt/docker:/opt/kubernetes:/opt/stedolan:/opt/garethr:/opt/go/bin:/opt/go/home/bin)
10:53:23 Warning: implicit hook found: lein-junit.plugin/hooks
10:53:23 Hooks are deprecated and will be removed in a future version.
10:53:23 Leiningen 2.9.1 on Java 1.8.0_232 OpenJDK 64-Bit Server VM
@balaji.sivaramgari Two possibilities come to mind: 1) the PATH for the process does not actually have lein
on it, despite the shell that you are running having it or 2) the process does not have sufficient memory allocated to it to run Leiningen.
Leiningen itself spawns two JVMs: one to run the initial command and then one to run the Clojure process that kicks off.
(when I was first trying to run Leiningen on early builds of WSL on Windows, it would fail with error=2
because it wasn't allocating process memory correctly)
sh "curl https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein > /tmp/lein" sh "chmod a+x /tmp/lein" sh 'echo "export PATH=$PATH:/tmp/lein" >> ~/.bashrc' sh "source ~/.bashrc" sh "echo $PATH" sh "/tmp/lein do cloverage" sh "/tmp/lein -v" this is the code snippet I used inside the jenkinsfile
the PATH should contain the folder the executable is in, not the executable itself
by calling /tmp/lein
you are ignoring the path and directly executing a file
if your path was correct you could just run lein
@U051SS2EU I have tried as below. But still it is not accepting the lein sh "curl https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein > /tmp/lein" sh "chmod a+x /tmp/lein" sh 'echo "export PATH=$PATH:/tmp" >> ~/.bash_profile' sh "source ~/.bash_profile" sh ". ~/.bash_profile" sh "echo $PATH" sh "cat ~/.bash_profile"
14:21:35 + cat /home/jenkins/.bash_profile
14:21:35 export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/alloy-jdk8-3.4.2/bin:/opt/docker:/opt/kubernetes:/opt/stedolan:/opt/garethr:/opt/go/bin:/opt/go/home/bin:/tmp
14:21:38 + lein do cloverage
14:21:38 /home/jenkins/workspace/Conflux_Clojure_Test@tmp/durable-cc1e0c9b/script.sh: line 1: lein: command not found
Please suggest, thanks in advance
@U051SS2EU this is happening in my local windows machine.
15:21:27.477 ERROR: Error during SonarQube Scanner execution org.sonar.api.utils.command.CommandException: http://java.io.IOException: Cannot run program "lein": CreateProcess error=2, The system cannot find the file specified at org.sonar.api.utils.command.CommandExecutor.execute(CommandExecutor.java:102) Caused by: http://java.io.IOException: Cannot run program "lein": CreateProcess error=2, The system cannot find the file specified at java.base/java.lang.ProcessBuilder.start(Unknown Source) C:\Users\sbalaji\Desktop\conflux>lein -v Warning: implicit hook found: lein-junit.plugin/hooks Hooks are deprecated and will be removed in a future version. Leiningen 2.9.1 on Java 1.8.0_231 Java HotSpot(TM) Client VM
when you run sh "source ~/.bash_profile"
that creates a new shell, makes it load the profile, and then it exits
it doesn't effect your shell
@U051SS2EU, yes we tried to add multiple shell commands in a single shell. Now it is working fine. We are able to push the metrics to sonarqube server.
glad I could help
11:05:07 + curl
11:05:07 % Total % Received % Xferd Average Speed Time Time Time Current
11:05:07 Dload Upload Total Spent Left Speed
11:05:08
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 12537 100 12537 0 0 44772 0 --:--:-- --:--:-- --:--:-- 44935
11:05:08 [Pipeline] sh
11:05:08 + chmod a+x /tmp/lein
11:05:08 [Pipeline] sh
11:05:08 + echo 'export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/alloy-jdk8-3.4.2/bin:/opt/docker:/opt/kubernetes:/opt/stedolan:/opt/garethr:/opt/go/bin:/opt/go/home/bin:/tmp/lein'
11:05:09 [Pipeline] sh
11:05:09 + source /home/jenkins/.bashrc
11:05:09 ++ export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/alloy-jdk8-3.4.2/bin:/opt/docker:/opt/kubernetes:/opt/stedolan:/opt/garethr:/opt/go/bin:/opt/go/home/bin:/tmp/lein
11:05:09 ++ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/alloy-jdk8-3.4.2/bin:/opt/docker:/opt/kubernetes:/opt/stedolan:/opt/garethr:/opt/go/bin:/opt/go/home/bin:/tmp/lein
11:05:09 [Pipeline] sh
11:05:09 + echo /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/alloy-jdk8-3.4.2/bin:/opt/docker:/opt/kubernetes:/opt/stedolan:/opt/garethr:/opt/go/bin:/opt/go/home/bin
11:05:09 /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/alloy-jdk8-3.4.2/bin:/opt/docker:/opt/kubernetes:/opt/stedolan:/opt/garethr:/opt/go/bin:/opt/go/home/bin
11:05:09 [Pipeline] sh
11:05:10 + /tmp/lein do cloverage
this is the output in the jenkins build log
you should pick your databases accordingly to the problem you are solving , not have a favorite one 🙂
Anyone else deal with Connection refused: connect
issue when running chromedriver and etaoin?
@U45T93RA6 ya i checked there to see if this happened to anyone else, no solutino yet
Is it not possible to use “str” to produce a querystring in jdbc? I’ve had issues with it, but only when using “str” with only elements of type string.
using str to make SQL strings is a path that can lead to sql injection attacks
@hiredman @alexmiller I was using a combination, the prepared statement was only modified by me, just tried some magic with pre-processing the statement, before using the standard prepared statement.
and then injecting the user values with “?”
might just be poor design on my part 😅 @alexmiller
luckily the user values are handled correctly
is (read-string "`1") being 1 and not (quote 1) an optimization? (i see the lines for it in LispReader.java, am curious about reasoning.)
What use would the quote be?
I would say no, it’s not an optimization, it’s an important part of the definition of Clojure
The reader takes strings and returns Clojure data. The number 1 (a Java Long) is the data representation of integers.
i have been confused trying to understand quote and syntax-quote -- in many ways they are similar, but (read-string "'1") gives (quote 1)
‘ is a special syntax that expands to the special form (quote <val>)
The quote special form, when evaluated, returns it’s value without evaluating it
thanks for trying -- i am still failing to escape my confusion -- i will keep reading the source and experimenting
Why is the quoted return from reading ‘1 confusing to you?
Perhaps it’s that ’1 expands to (quote 1), while `1 doesn’t expand to, for example, (syntax-quote 1)?
I have a huge data structure with maps and vector of maps. Everything nested. I have to find path to the key :foo
. It is so big I can’t figure out manually how to read this key. I have to read many keys like that.
So there is {:a {:b [{:c 1} [{:d 1}]}}
etc. imagine it much more deeper and nested.
I need to find a way how to read :foo
from it. So I have to figure out whole path to it: Like :a
:b
map of vector
key :c in each map in the vector
and deeper and deeper.
Any tricks how can I find the path?
Sounds like a job for https://github.com/redplanetlabs/specter
Depends on what you mean by "find path to the key". It can mean either of:
1. Knowing at what "layer" :foo
keys are used (like [:a :b *all-vectors* :c :foo]
, navigate to all of them and get all of the values
2. Not knowing where :foo
keys are used at all, find all the paths to them.
If it's #1, then it's definitely select
.
#2 would be a bit more complicated but it's still possible. Although the end result probably wouldn't be much shorter than if you'd use raw Clojure with no libraries. About 10 lines or so.
I need path to :foo
to write code which read :foo from this structure. It is just so complex I can’t figure out the path manually
YAML is not a great idea. 🙂 Even by itself, the format is quite bad.
I'm still not sure what you want. Suppose you already have this magic function f
- what are the inputs and what is the desired output? Can you give a small example just so I'm sure I understand?
A small example based on specter
of something that seems to be similar to what you want:
(select [:a MAP-VALS ALL :b]
{:a {:x [{:b 1}
{:b 2}]
:y [{:b 3}]}})
=> [1 2 3]
I have :foo
somewhere if very complex structure. I have to figure out the path to :football:
So you have to figure it out to write code to read this later like (get-in [:a :fsda :fsda :fhuifader :fsdfgyuasdf :sdfafasdsd]) and more and more and pass through vectors of maps
Is :foo
at a fixed place? Or are these keys placed at multiple different locations throughout the structure?
OK, so you can easily use get-in
once you figure out where it's at. The only task is to find this single needle in a haystack.
well some kind of get-in
with map
because :foo
can be multiple times, but always in the same path (considering N maps in vector)
> can be multiple times, but always in the same path (considering N maps in vector)
Well, that's different paths. Vectors are exactly like maps from 0-based indices to the values for our purpose.
OK, so lists as well. Anything else?
If :foo
is in the fist map within a vector, can we safely assume that the rest of the maps have this key as well?
(def data {:a {:b {:c [{:x 1}
{:foo 2}
{:yy 7}]}}})
(defn find-first-key-path [d tk]
(letfn [(get-path [k v]
(when-some [p (find-first-key-path v tk)]
(into [k] p)))]
(cond
(map? d) (if (contains? d tk)
[tk]
(first (keep (fn [[k v]] (get-path k v)) d)))
(sequential? d) (first (keep-indexed get-path d)))))
(find-first-key-path data :foo)
If so, finding a path to the key is not possible without returning the whole map from a set since sets are not indexable.
Other possibilities: 1. The keyword is not used as a key of a map. It maybe a first item in a two-item list in a vector or something like that. Or just a keyword in a vector. 2. The keyword is just not there. 🙂
I don't think there are any other ways for the find-first-key-path
function to fail to find something.
OK. And is it really so huge that you can't possibly pretty-print it, find the key with the regular text search, and go back from there?
Ah, another possibility - if the map with this key is itself used as a key, the function above will not find it.
Wait, so you can open the data in an editor. Why can't you not pretty-print the data then?
also I found yesterday key can be in a few different places with a different meaning
Ah, sorry, I read one of the previous messages wrong - I thought you said that you can't write data to the file.
1. Load the data in REPL 2. Pretty-print it, but not into REPL itself but to a string 3. Write that string to a regular text file (or just write to a file right away and skip the string altogether) 4. Open the file in some editor/viewer 5. Find the required key with a regular text search 6. Go from the bottom to the top and from the right to the left while gathering all elements of the path to the required key
In order for this to be insurmountable for manual work, either the key has to be thousands of levels deep, or the data has to be multiple GBs in size.
Yet another way to handle it - try to convert it to JSON. The keywords will of course become strings. When the conversion fails, understand why and fix it - e.g. turn all numeric keys into strings, turn all sets into vectors, remove all custom types that can't have any content, turn all custom types that can into maps etc.
You will end up with a regular JSON file. When you highlight the required key, IDEA (and I'm sure other IDEs/editors as well) will show you the breadcrumbs to it, which is exactly what you need.
yeah, converting it to JSON in the one way. I was trying, but give up, because of magic errors during it 🙂
OK, I have no idea how you can't find it then. 15MBs with 10 level nesting is nothing. You said you cannot send the data. But can you show it on your screen? If you want, we can hop on a call and you can share your screen so that I could see what you're doing and what prevents you from finding the path to the key.
there is a lot of mess, because of values itself. If I could see only keys without values hmm
Hmm maybe we can try screen share, but now I am a little in rush. I can do it +6 hours from now or something like that.
> You cannot, because the key is in one of the values. I mean by remove last leaf in the tree, so the values.
Thank you, that is silly and interesting issue. I appreciate your engage into the topic. Of course I don’t want to take too much your time 🙂
It won't achieve much, since the leaves can be only of primitive data types. "Vector of keywords" is not a leaf.
in syntaxQuote (LispReader.java), some of the cases essentially end up as (quote form) -- so superficially, quote and syntax-quote appear to have similar behavior. for some reason, keywords, symbols, numbers, and strings don't. yet those 4 things when wrapped in (quote ...) and passed to read-string, do end up with (quote ...) around them. that seems somehow odd to me.
I would flip your sense here and see syntax quote as the weird case
that was my initial impression -- which was why i asked whether it was an optimization 🙂
Well you didn’t ask in the context of syntax quote
Those are all things that evaluate to themselves
except symbols (I almost said the same thing :D )
Right, thx
So the quote is unnecessary in those cases and could be seen as an optimization
The symbol is more about the semantics of syntax quote
just to be clear then, apart from the symbol then (still thinking about that one), the syntaxQuote behavior for strings, keywords, and numbers might be seen as an optimization then?
Well I think if it was quoted, you’d be unhappy looking at the results too (lots of unnecessary quoted stuff) even if it eval’ed the same
I probably spend an unhealthy amt of time looking at macro expansions though
i don't disagree that verbosity would increase -- it was just surprising as i was trying to study some macro expansion things
(on a side note, i tried compiling a custom version of clojure with the quote being thrown in and it appears to have passed all tests)
I think the truth is it's just like that. Maybe it was an optimization, maybe it was an accident. But in any case, it doesn't seem to matter much.
I think it could also be that syntax-quote isn't about quoting things, but about fully qualifying them and performing templating over the input