Fork me on GitHub
#clojure
<
2023-03-14
>
tengstrand07:03:26

I have problems reading Clojure source files that contains reader conditionals. I have no problems with all other “standard” Clojure files. The problem is explained in my comment at the end of https://github.com/polyfy/polylith/issues/259 issue.

p-himik07:03:22

read has a :read-cond option.

tengstrand08:03:33

Okay, thanks, will check that out!

borkdude08:03:20

Note that clojure's read doesn't let you control the features. For this it's probably better to use tools.reader or edamame: https://github.com/borkdude/edamame#reader-conditionals

borkdude08:03:18

Oh, wait read does let you control the features, but there was something edge-casey with this: in Clojure's read you can't disable the :clj feature or so

borkdude08:03:10

user=> (read-string {:read-cond :allow :features #{:cljs}} "#?(:clj 1 :cljs 2)")
1

tengstrand08:03:45

Perfect! Thanks.

borkdude08:03:22

No, that wasn't perfect. Check the difference.

$ clj -Sdeps '{:deps {borkdude/edamame {:mvn/version "RELEASE"}}}'
user=> (require '[edamame.core :as e])
nil
user=> (e/parse-string "#?(:clj 1 :cljs 2)" {:read-cond :allow :features #{:cljs}})
2

tengstrand08:03:21

Okay, will try it out!

borkdude08:03:31

If you just want to read all the forms and you don't care about alias resolving, you can do this:

user=> (e/parse-string-all "[1 2 3] #?(:clj 1 :cljs 2)" {:read-cond :allow :features #{:cljs}})
[[1 2 3] 2]

borkdude08:03:06

but not 100% sure how you are using the reader for polylith

tengstrand08:03:06

Will fiddle with it, but thanks.

borkdude09:03:15

Your read-file will currently only partially read this file:

(ns foo (:require [clojure.set :as set]))
::set/foo
123
It will only read the first ns form since it will crash on the second form, but maybe that's your intention.

tengstrand09:03:54

No, that’s not my intention. But users haven’t reported this as a problem. If the file only contain def/defn/defmacro + comments, then it works. But I guess it would be good if it doesn’t crash in the example you give.

borkdude09:03:57

Here is an example how you can parse ns forms more accurately and then resolve to the right keywords, etc https://github.com/borkdude/edamame/blob/master/examples/auto_resolve.clj If you don't care about the "right" keywords, but keep reading, you can just do this:

user=> (e/parse-string "[::foo ::str/foo]" {:auto-resolve name})
[:current/foo :str/foo]

tengstrand09:03:41

How would the code look like if I want to read from a file? (I can probably figure that out, but sometimes it’s tricky)

tengstrand09:03:14

I mean, should I read the whole content of the file and convert it to a string and then do e/parse-string, or should I read the file as a stream or something (haven’t looked in the doc yet).

borkdude09:03:57

you can do two things: first create a reader with e/reader and then call e/parse-next on it repeatedly, or slurp the file and then call e/parse-string-all

tengstrand09:03:40

It worked perfectly! Thanks again for the help (I used e/parse-string-all).

partyparrot 4
tengstrand07:03:35

I configured parse-string-all https://github.com/polyfy/polylith/blob/6214667cf1493f57f19cace122c761bd1fc85a42/components/file/src/polylith/clj/core/file/core.clj#L110-L121 way and I’m now able to read the whole Polylith workspace with it. You may take a quick look @U04V15CAJ to see if I got it right! It’s important that I can read all variants of valid .clj source code, so maybe there are more options I need to add (or maybe some I should remove).

borkdude08:03:46

I’ll take a look later today

👍 2
borkdude09:03:57

@U1G0HH87L This looks ok. With the newest edamame you can also use :auto-resolve-ns for better keyword reading like ::alias/foo . What are you doing with the results? For closer clj defaults, also check here: https://github.com/borkdude/edamame#clojure-defaults

tengstrand11:03:06

I’m extracting the :require statements from the ns for .clj files + reading .edn files (which reading everything from). I added :auto-resolve-ns to the list of attributes and it worked!

borkdude11:03:25

@U1G0HH87L Ah then you are in luck, since edamame now also has a edamame/parse-ns-form function which returns the :requires

👍 2
borkdude11:03:33

user=> (e/parse-ns-form '(ns foo (:require [foo :as bar])))
{:current foo, :meta nil, :requires ({:as bar, :require true, :lib foo}), :aliases {bar foo}}

borkdude11:03:12

It also supports gnarly edge cases like nested libspecs:

user=> (e/parse-ns-form '(ns foo (:require [foo [bar :as bar]])))
{:current foo, :meta nil, :requires ({:as bar, :require true, :lib foo.bar}), :aliases {bar foo.bar}}

tengstrand11:03:06

Sounds great.

mesota09:03:05

Is there Fulcro-like RAD tool to be used in luminus stack? Something like code generation based on data model for things like CRUD

javahippie13:03:08

We started building something like this, but are nowhere near it being useful for our one usecase. The problem I see with it for the Luminus stack is, that it roughly can pull in everything. Different SQL drivers, XTDB, Datomic, spec, malli, reitit, hiccup, selmer, re-frame…

👍 2
chrisn12:03:59

Is there a way to disable mutlthreaded artifact resolution when getting dependencies of a deps.edn based project? I am getting lock errors:

(base) chrisn@chrisn-lp2:~/dev/tech.all/tech.opencv$ clj
Downloading: org/bytedeco/opencv/4.6.0-1.5.8/opencv-4.6.0-1.5.8-ios-arm64.jar from central
Downloading: org/bytedeco/openblas/0.3.21-1.5.8/openblas-0.3.21-1.5.8-windows-x86_64.jar from central
Error building classpath. Could not acquire write lock for 'artifact:org.bytedeco:opencv:4.6.0-1.5.8'
java.lang.IllegalStateException: Could not acquire write lock for 'artifact:org.bytedeco:opencv:4.6.0-1.5.8'
        at org.eclipse.aether.internal.impl.synccontext.named.NamedLockFactoryAdapter$AdaptedLockSyncContext.acquire(NamedLockFactoryAdapter.java:165)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:233)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:212)
        at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveArtifact(DefaultRepositorySystem.java:272)
        at clojure.tools.deps.alpha.extensions.maven$get_artifact.invokeStatic(maven.clj:161)
        at clojure.tools.deps.alpha.extensions.maven$fn__1152.invokeStatic(maven.clj:173)
        at clojure.tools.deps.alpha.extensions.maven$fn__1152.invoke(maven.clj:169)
        at clojure.lang.MultiFn.invoke(MultiFn.java:244)
        at clojure.tools.deps.alpha$download_libs$fn__799$fn__800.invoke(alpha.clj:464)
        at clojure.tools.deps.alpha.util.concurrent$submit_task$task__481.invoke(concurrent.clj:35)
        at clojure.lang.AFn.call(AFn.java:18)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)

ghadi12:03:57

try -Sthreads 1

👍 2
chrisn13:03:46

Thanks for such a quick reply! That is worst issue. Other issues is that it downloads all jars not just the jars for my current platform.

2
ghadi13:03:32

for that second issue, what is a maven coordinate that demonstrates that?

chrisn13:03:24

org.bytedeco/opencv-platform {:mvn/version "4.6.0-1.5.8"}

chrisn13:03:41

Last issue opencv only has a c++ interface making dynamic binding via jna and friends impossible leading to issues like this 🙂

Carsten Behring16:03:58

I just opened an AskClojure issue with it. https://ask.clojure.org/index.php/12730/error-could-acquire-write-lock-artifact-org-bytedeco-opencv It was logged in Jira. Interestingly it happened only on "slow network"

Jakub Šťastný23:03:25

How do I install latest Clojure CLI on Linux without adding external dependencies (Linuxbrew)? Everything I can found is post-fixed with the version, such as https://download.clojure.org/install/linux-install-1.11.1.1252.sh, but I cannot find the current version anywhere predictable. For instance https://raw.githubusercontent.com/clojure/clojure/master/pom.xml has the version, but it's the master branch, so the version is 1.12.0-master-SNAPSHOT. I need either: • A URL that never changes, as in an (official) install script that doesn't include the version if there's one. • A way to get the version from somewhere, so I can correctly construct (str "linux-install-" version ".sh").

pppaul23:03:03

I use arch, clojure cli should be up to date and can be installed via pacman

Jakub Šťastný23:03:54

@U0LAJQLQ1 I like Arch, I've been using it for a good time back in the day when I was using Linux as my main OS.

Jakub Šťastný23:03:02

These days I use Ubuntu for my images though.

pppaul23:03:09

if you are using docker, you probably do want to specify the version number for clojure cli in your build process

Jakub Šťastný23:03:26

This is Ubuntu that will be released in April and has Clojure 1.10.

pppaul23:03:34

Ubuntu is hell

Jakub Šťastný23:03:42

I'm using Docker actually.

pppaul23:03:08

I use Ubuntu on my servers, and I'm pretty sure it's one of the main reasons why docker is popular

Jakub Šťastný23:03:38

I wanted not having to specify (I keep forgetting). The idea is get me latest Clojure, don't care which it is as long as it's the latest one.

pppaul23:03:25

if you are using docker, then hard coding a version of a dep is ideal

phronmophobic23:03:08

Before macports supported tools build, I used to run the install script used by the homebrew tap manually, https://download.clojure.org/install/clojure-tools-1.11.1.1252.tar.gz

pppaul23:03:32

I think to get the latest version of something on Ubuntu, maybe you can use snap, or a ppa and use apt, but you can never be sure that those will give you up to date versions

Jakub Šťastný23:03:50

@U0LAJQLQ1 what do you mean by hard-coding? Setting it as ENV at the beginning of the script? I have it hard-coded, but somewhere in the middle 🙈 https://github.com/jakub-stastny/dev/blob/literate.dev/build.org#clojure-cli

Jakub Šťastný23:03:12

Jesus now there are so many bloody package managers.

Jakub Šťastný23:03:57

@U7RJTCH6J cheers man! The version's right there, that'll work!

pppaul23:03:02

there is flat pack too, and yes there are way too many package managers on Ubuntu, and they are all bad

pppaul23:03:51

for hardcode, I just mean in your docker, if you build it a year from now that you would still build with the same version of clojure

Jakub Šťastný23:03:11

On Linux in general. It's always the main + Linuxbrew/Nix, then all the new Ubuntu ones or what...I don't really follow, it just is a new thing every time and it's too much!

seancorfield23:03:26

Use the script name without the version.

pppaul23:03:41

nix is the now hotness, but it's actually pretty old

seancorfield23:03:06

FWIW, https://github.com/clojure/brew-install/blob/1.11.1/stable.properties contains the version (and SHA) for the latest 1.11.1.x version in case that's usefl.

clojure-spin 2
seancorfield23:03:58

I'm adding a Getting Started with Clojure CLI to clojure-doc that mentions these scripts https://clojure-doc.org/articles/tutorials/getting_started_cli/ (it's a work in progress right now)

💙 4
💯 2
Jakub Šťastný23:03:33

@U04V70XH6 thanks, that's very useful. Going to check it later 🙂

seancorfield23:03:19

We have a script at work that starts off:

#!/bin/sh
if test "$1" = ""
then
  echo "Usage: update-clj.sh <version> [preview]"
  echo "-- preview is required if version is not current stable release"
  exit 1
fi
version=$1

curl -O 
read stable_version stable_sha < stable.properties
rm stable.properties

if test "${version}" = "stable"
then
  version=${stable_version}
fi
...
We use this to update the vendored version we keep in our repo (so every developer/environment can rely on the same CLI version easily).
sh build/bin/update-clj.sh stable
Updates us to the latest stable version, whatever it happens to be. We can also specify an explicit version and even move to a non-stable version by adding preview as the second argument. https://github.com/clojure/brew-install/blob/1.11.1/devel.properties has the current preview version (which, right now, is the same as the stable version).

Alex Miller (Clojure team)00:03:03

https://download.clojure.org/install/stable.properties is maybe an easier url to get latest stable version info but the versionless urls above are supported

Alex Miller (Clojure team)00:03:54

(the github 1.11.1 will break when we re-version, which is its own issue to fix some day)

seancorfield05:03:50

(yeah, our script at work uses that "easier url" for the stable version -- I should have specified that in my comments, rather than the links, thanks!)