This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2024-01-29
Channels
- # announcements (1)
- # beginners (176)
- # biff (3)
- # calva (7)
- # clojure (68)
- # clojure-europe (18)
- # clojure-nl (1)
- # clojure-norway (12)
- # clojure-uk (6)
- # community-development (4)
- # conjure (1)
- # core-async (5)
- # datomic (21)
- # events (1)
- # fulcro (5)
- # funcool (3)
- # hyperfiddle (35)
- # leiningen (18)
- # malli (3)
- # nbb (20)
- # overtone (20)
- # pedestal (1)
- # polylith (68)
- # portal (6)
- # releases (1)
- # shadow-cljs (6)
- # slack-help (7)
- # squint (6)
- # vim (4)
- # xtdb (4)
Using leiningen, is there some way to use SHA256 checksums for dependencies? Specifically: I'm trying to use a package from maven central, but these seem to use sha256 for checksum? https://repo1.maven.org/maven2/com/microsoft/cognitiveservices/speech/client-sdk/1.34.1/ This results in a "Checksum validation failed, no checksums available". Is there any way to get this to work (short of disabling checksum validation in project.clj)?
I'd like to read from a config.edn
file that should be in the same directory as the jar. So, I'm looking for a way to get the path to the jar. The following snippet (https://stackoverflow.com/a/13276993) works when running the jar, but does not compile at the REPL (`ClassNotFoundException` for the foo.core
symbol in -main
):
(ns foo.core
(:gen-class))
(defn this-jar
"utility function to get the name of jar in which this function is invoked"
[& [ns]]
;; The .toURI step is vital to avoid problems with special characters,
;; including spaces and pluses.
;; Source:
(-> (or ns (class *ns*))
.getProtectionDomain .getCodeSource .getLocation .toURI .getPath))
(defn -main [& _]
(println (this-jar foo.core)))
Yeah I wouldn’t trust that the namespace object would necessarily lead to the right classloader
This whole scheme seems fraught with problems though - can you just include in the classpath and load as a resource?
I'm distributing the jar to end users and they should be able to edit the config file.
I thought about e.g. ~/.config
or similar, but being in the same dir as the jar would be cleaner.
Being in the same dir as the jar is going to be problematic for most users
Java has the concept of resources loaded from the classpath and every Java or Clojure dependency tool lets you manage that
But I would recommend giving it a better path and name
As you are sharing that space with the rest of the world I would call it foo/core/config.edn etc
https://clojuredocs.org/clojure.java.io/resource is what you can use to load it
I haven't tried on jars but what about (System/getProperty "user.dir")
? That seems to return the PWD on windows and linux
I guess that only works if you run the java -jar for the same dir as the config
> Being in the same dir as the jar is going to be problematic for most users
Could you explain why?
> As you are sharing that space with the rest of the world I would call it foo/core/config.edn etc
What is "that space" referring to? The jar dir? I guess I imagined the jar would be in a dir like .../app-name/...jar
, so that config.edn
wouldn't be confusing. But I see your point about jar itself. However, I'd also want the database files to be in the app dir, so I guess an app dir should be a given.
> I guess that only works if you run the java -jar for the same dir as the config Yup 🙂
I also thought io/resource
cannot be used to load files outside the classpath?
(Complete Java ignoramus here :))
Generally people manage jar files by using a dependency manager that stores them in a maven cache. This is a per user cache, managed by maven, and is not a place you should expect users to put files
If you need a place for users to manage files, that should really be something that they tell you, or you should establish some convention about where to put it
“Dir of the jar” is a bad answer
I'm looking to distribute a full stack web app (#brimm). Currently I'm going with providing a jar, wrapped with https://github.com/ericdallo/deps-bin to be slightly friendlier to non-technical users.
So I think the convention "all user files live in a .../brimm/
" dir is fairly reasonable?
If you’re controlling the final package then you may have more to say about it. You might to consider making an executable with Graal etc
Graal is on the todo list. I'll also have to think about people using Brimm as a clj library. Anyway, thanks for your input!
I'm trying to use some plotting software (from https://github.com/metasoarous/oz) I added [metasoarous/oz "2.0.0-alpha5"]
to my :dependencies
in my project.clj
file as suggested by the page https://github.com/metasoarous/oz
Then I ran lein test
from the shell to see if everything was working.
it triggered 613 lines to be printed like the following.
Retrieving io/forward/yaml/1.0.9/yaml-1.0.9.jar from clojars
Retrieving ring/ring-devel/1.8.1/ring-devel-1.8.1.jar from clojars
Retrieving ring/ring/1.8.1/ring-1.8.1.jar from clojars
Retrieving ns-tracker/ns-tracker/0.4.0/ns-tracker-0.4.0.jar from clojars
Retrieving ring/ring-jetty-adapter/1.8.1/ring-jetty-adapter-1.8.1.jar from clojars
Retrieving ring/ring-servlet/1.8.1/ring-servlet-1.8.1.jar from clojars
Retrieving bk/ring-gzip/0.3.0/ring-gzip-0.3.0.jar from clojars
Retrieving carocad/parcera/0.11.6/parcera-0.11.6.jar from clojars
and now when I restart cider, I see the following troubling warnings:
WARNING: abs already refers to: #'clojure.core/abs in namespace: clojure.test.check.generators, being replaced by: #'clojure.test.check.generators/abs
WARNING: abs already refers to: #'clojure.core/abs in namespace: medley.core, being replaced by: #'medley.core/abs
WARNING: update-vals already refers to: #'clojure.core/update-vals in namespace: clojure.tools.analyzer.utils, being replaced by: #'clojure.tools.analyzer.utils/update-vals
WARNING: update-keys already refers to: #'clojure.core/update-keys in namespace: clojure.tools.analyzer.utils, being replaced by: #'clojure.tools.analyzer.utils/update-keys
WARNING: update-vals already refers to: #'clojure.core/update-vals in namespace: clojure.tools.analyzer, being replaced by: #'clojure.tools.analyzer.utils/update-vals
WARNING: update-keys already refers to: #'clojure.core/update-keys in namespace: clojure.tools.analyzer, being replaced by: #'clojure.tools.analyzer.utils/update-keys
WARNING: update-vals already refers to: #'clojure.core/update-vals in namespace: clojure.tools.analyzer.passes, being replaced by: #'clojure.tools.analyzer.utils/update-vals
WARNING: update-vals already refers to: #'clojure.core/update-vals in namespace: clojure.tools.analyzer.passes.uniquify, being replaced by: #'clojure.tools.analyzer.utils/update-vals
WARNING: abs already refers to: #'clojure.core/abs in namespace: taoensso.encore, being replaced by: #'taoensso.encore/abs
WARNING: update-keys already refers to: #'clojure.core/update-keys in namespace: io.aviso.exception, being replaced by: #'io.aviso.exception/update-keys
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
Is this what I should expect? or does it seem strange?
The Retrieving yes - that’s downloading all the transitive deps, which are cached in ~/.m2
The warnings are mostly from loading older versions of these deps, I think most of those are fixed in newer versions
so should I remove ~/.m2 and try again?
or am I using the wrong version of metasoarous/oz ?
looks like metasoarous hasn't been touched in years. maybe I'm must using the wrong plotting interface?
I can’t answer that, but nothing is wrong here, these are just warnings
> looks like metasoarous hasn't been touched in years. maybe I'm must using the wrong plotting interface? These are just warnings, and Oz probably works fine. I think these days a lot of people make their way to vega-lite in clojure with https://github.com/nextjournal/clerk/.
No need to remove m2, the warnings are fine, they occur when a ns defines a var that's also defined in clojure.core. happens in old libraries. But oz works fine
so when my newly students start up Clojure for the first time in this project, they'll see all these warnings. that will certainly be confusing. no?
They'll also see a very long list of jars being downloaded.
You can just forewarn them what they'll see being logged and tell them not to get excited
Why is a warning a greater cause for alarm than SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
?
yes failure to load class sounds like a real problem to me.
Exactly. However, in this case, it's just noise. So what do you tell your students? Don't panic, and if something doesn't work, don't try to guess what broke it, send the entire log
I here loads about ring, but I wondered if there is a lighter weight webserver tech that can handle routes etc, but does not need all the options of ring?
I don't understand the question. Ring is just a spec for glue-libraries that help idiomatic Clojure web-apps use existing web-server libraries like Jetty. Jetty alone is indubitably "lighter" than Jetty + ring-jetty-adapter ... but Jetty alone is so big that the difference is insignificant.
Ring converts http requests and responses to/from a Clojure hash-map. Other libraries are used to route requests to handler functions A simple web server in Clojure https://practical.li/clojure-web-services/app-servers/create-server/
Hello guys. I'm having trouble understanding recur. I though I got it, but clearly not. I wrote count
like this:
(fn [x] (if x (+ 1 (recur (next x))) 0))
But it does not work. I looked at the solutions in 4clojure, and found:
(fn cnt [x] (if x (+ 1 (cnt (next x))) 0))
which does work. Why?Did you get an error?
recur
requires that it is called in the tail position. This means that the function has to return the value of the recur
form.
4Cloujure says, ""uh-oh". And when I run it with ctrl-return, I see this:
((fn [x] (if x (+ 1 (recur (next x))) 0)) '(1 2 3))
user=> 1:sci.impl.analyzer/recur
ah ok. that's not a very helpful error. the compiler error on the JVM is a bit more helpful.
I clearly need to go read what tail position is then, because I would define "the executable forms of the last if in a form as the tail positions". Especially, since when I first wrote my function, I was careful to have if's first form return 0 and the second one be the recur.
Yea, as @U0P0TMEFJ was saying, recur
must be called in "tail position".
From https://clojure.org/reference/special_forms#recur
> Note that recur is the only non-stack-consuming looping construct in Clojure. There is no tail-call optimization and the use of self-calls for looping of unknown bounds is discouraged. recur is functional and its use in tail-position is verified by the compiler.
In general, recursive calls are allowed anywhere, but recur
must happen in "tail position" so that it can avoid consuming the stack.
if
will return either x
or (+ 1 (recur (next x)))
- those are the two forms in the tail position of your function
So what I originally wrote, was this:
(fn [x] (if (not (next x)) 0 (+ 1 (recur (next x)))))
Which, to me, has recur in the tail position. (I learned about the tail recursion optimization 20 years ao in Scheme, and I feel confident I understand it, but this example is showing me I don't understand it like Clojure does.)that's where you need an argument to your function that is the current value of the sum, and you increase it for each element of the list
the tail position is where no work is done at the current stack frame, so it can be optimised away
OK, here's another solution for mthe problem 22 solutions page:
#(loop [i % c 0]
(if (not (seq i))
c
(recur (rest (seq i)) (inc c))))
I don't see the difference with respect to mine.((fn c ([x] (c 0 x))
([acc x]
(if x
(recur (inc acc) (next x))
acc)))
[1 2 3])
here, we have an accumulator that we just returnThe two possible tail call positions in
(fn [x]
(if (not (next x))
0
(+ 1 (recur (next x)))))
are
0
and
(+ 1 (recur (next x)))
. Recur is not the tail call, (+ 1 (recur ,,,))
is! Recur must be in a tail call position in order to do the magic it does — increase performance.ok, teo that makes lots of sense. I was drilling inside the + to go find my idea of why it should work.
imagine it like you call the recur
function, and directly return the value from that. If you have to do more work (like (+ 1 ,,,)
) then it's not in the tail position
And with an if, with it's two tail call sites, only one of the two could be used, right?
You can write an if
where both 'then' and 'else' branches have recur
inside of them, but there must be some branch somewhere that doesn't recur, or you will get infinite recursion.
(And the answer to my original question, why does it work when named instead of unnamed, I now completely understand too. yay!)
cool. So you only get tail call optimisation with the recur
form. if you use a named function you will be consuming the stack 😉
I have a fairly long transducer pipeline that may throw an exception in the middle. What's the idiomatic way catching an exception within a transducer?
It depends. Is it a known exception? Is it recoverable? It's hard to give an answer without knowing more about the use case.
It's not recoverable, so I need to restart the transducer but I have some transient state within the transducer I also want to save out
> I have some transient state within the transducer I also want to save out Most transducers don't allow you to do that. > I need to restart the transducer Are you trying to resume from where the error occurred? What do you mean by restart?
Yes I am wanting to resume. The specific case is I am doing lots of http calls and they can fail
There's a couple options. How are you running the pipeline? transduce? iteration?
transduce
Are you interested in adding retries? Do you need to keep track of which requests failed?
There are retries already but it can fail due to other reasons such as a service outage. I would like to keep track of what failed
The nice thing about transducers is that you can break things apart into simpler pieces. Some options:
• You can use something like https://github.com/cgrand/xforms to write fancy transducers that "fork" after the http call part of the pipeline. let's say you have a (map http-call)
as one of the transducers. You can have the result either indicate success or failure. You can then use something like transjuxt
to handle the successes and failures separately
• You can write your own transducing context that handles errors however you would like
Another option is to use something like sequence
or iteration
to turn your pipeline of http calls into data and then run analytics on the success and log the errors. This is the approach that I've used.
I'm not saying this is the best code, but it's at least an example, https://github.com/phronmophobic/dewey/blob/main/src/com/phronemophobic/dewey.clj#L141 This project scans the github api for clojure libraries. My retry code simply aborts if retries don't work, but you can could return an error value. The data from the API is written to disk, which is processed separately at a later point. You could instead process it immediately in memory if you wanted. The nice thing about writing it to disk is that you're separating the I/O from the data transformation.
I guess what I'm advocating is breaking your pipeline down into at least two steps: 1. collecting data 2. analyzing the data These correspond to the E and the T of https://en.wikipedia.org/wiki/Extract%2C_transform%2C_load.
Thanks for all the links! When you say creating my own transducing context are you saying I should make a new function (like into
) and have it do something to help with this?
I actually use iteration
extensively throughout the transducer, however I process everything as soon as one item is ready rather than making all the http calls first
> Thanks for all the links! When you say creating my own transducing context are you saying I should make a new function (like into ) and have it do something to help with this? The options of creating your own context is about splitting apart the pieces of your pipeline and composing it with cross cutting concerns like progress monitoring, logging, error handling, and retries.
I actually use iteration extensively throughout the transducer, however I process everything as soon as one item is ready rather than making all the http calls firstYou can logically separate the steps of collecting data and analyzing data while still interleaving their sequence of execution.
It's really hard to give more specific advice without more context. There are lots of different options: • scaling horizontally • scaling vertically • threads vs goroutines • latency vs throughput • memory vs disk space vs networking vs I/O • etc. which depend on a whole host of factors: • ops environment ( cloud provider, dependencies, scale) • data volume (eg ops/second) • data scale (eg. bytes/second) • time budget • money budget • team budget • existing infrastructure • etc.
however I process everything as soon as one item is ready rather than making all the http calls firstGoing back to this point. Unless you have good reasons (which you may have), I highly recommend completely separating collecting the data and processing the data. It makes life a lot easier and is a common practice (eg. https://en.wikipedia.org/wiki/Lambda_architecture).
There are also lots of clojure and JVM libraries available in this space.
Yeah to be fair I really just have this big ass transducer because I thought it would be cool to have one function that did EVERYTHING (and I was also learning transducers) but it definitely has these problems. I think there isn't much separation between the collecting and processing in my case, it really is just hitting http endpoints, getting the json from the request and smacking it into a database
it really is just hitting http endpoints, getting the json from the request and smacking it into a databaseThat's totally fine for a lot of use cases. If things start getting more complicated or you want to make your pipeline more robust, one of the tools for doing that is breaking it down a bit so you can reconfigure the pieces. Just to give one simple example:
(require '[net.cgrand.xforms :as x])
(def xform (map (fn [n]
{:n n
:success? (even? n)})))
(def colls (range 20))
(def http-responses
(eduction
xform
colls))
(def pipeline-xform
(comp (map :n)
(map -)))
(def results
(x/transjuxt
{:error (comp
(remove :success?)
(x/into []))
:success (comp
(filter :success?)
pipeline-xform
(x/into []))}
http-responses))
obviously, your http-responses would be http-calls with retries rather than a collection. This will still process results as they're available, but the data collection and data transformation are logically separated.
Is there a better way to inject a logger with a built-in request context than the below? The issue with this implementation is that every log line is coming from this function so I can't know which place in the code actually logged it
(defn request-context
[handler]
(fn [request res raise]
(let [trace-id (.toString (random-uuid))
logger (partial make-logger trace-id)]
(logger :info
(str "Received request: " (pprint request)))
(-> (assoc request :trace-id trace-id :logger logger)
(handler res raise)))))
What logging system are you using?
Ah, can't help with that, sorry.
Does this channel also help with other Clojure-flavored languages ? I am working with Phel with is a Clojure-Flavored Lisp that compiles to PHP. I tried their Gitter channel but get no response. If this isn't the place, can someone point me in the right direction where I can get help with other Clojure-flavored langs ? Thank you !
I think @U02AM852K5F is the only person in this Slack to have mentioned Phel so maybe he can point you to the best place to get help? (he's based on Tokyo, so be aware of that timezone)
I don't know. I've never used it and I've not used PHP since 2005, but what's your question? If it's a beginner to Clojure type question, we might be able to help.
@U05N5T43AF5 Yeah, you might as well ask: if it's a language question, rather than an implementation question, chances are good that we can try to answer it here... I'm happy to run the Phel Docker image to try stuff out in the REPL for you too 🙂
Thank you seancorfield & Ed ! My question was that I am able to get everything up on running based on their "Getting Started Guide", but there is a specific error I cannot get rid of when pulling up the localhost/index.php page. I guess my question is how I would get rid of it ?
Everything in boot.phel works ! But I just have that error that I cannot get rid of for some reason.
ah ... ... sorry ... saw the orange background and assumed that's what you meant by "this error"
What is in hello-gordo\vendor\phel-lang\phel-lang\phel-config.php
And are you sure you have the appropriate version of PHP installed/running?
It says 8+ when I started the php server in CMD, which is what was required in the Getting Started guide.
I'm not much of a windows user, but there are both forward and backward slashes in the path - is that a problem?
You know Ed, IT MIGHT be becasue I notice that there are both forward and backward slashes in the error, I just don't know WHICH line I should be correcting for that (if I need to.).
Yes, I am using Windows 10. So should I change all the slashes in the phel-config file to backslashes ?
Okay, I changed all of the / to backslashes and still the same error. I even restarted the PHP server and refreshed but no bueno. Lemme try the double backslashes.
I'm not sure how file paths are handled by php on windows - I've not tried using windows as a server since asp classic
maybe change the config file to something like
$c = (\Phel\Config\PhelConfig())
->setSrcDirs...rest of the code
var_dump($c)
return $c
??That's okay ! I'll try something other time, I don't wanna burn your guys brain. How about this .... what would you suggest for Lispy web dev building ?
https://phel-lang.org/documentation/configuration/ ... you have some options that are not documented here
What I am trying to do is build web apps or SPAs with a LISPY language (prefereably Clojure)... and I thought Phel would nve a nice one to do xD
I just got a little overwheled at looking at all the tools (hiccup, Re-Frame Babashka etc)
i only know really html CSS, a little PHP and MySql But my Clojure skill has built FAST because its SO fun to code in. So thats why I figured to use a Clojure-flavored PHP like Phel
I think the learning experience is going to be a tough on with Phel. I think it's a bit more niche.
Okay how about thjs question.... what would you suggest for someone like me who wants to build a website that can pull data from a MySql (or a database in general) and display it on the page ?
Like, lets say I have a list of games I am collecting and I want to insert the game data in a database and pull it to display on the page. What tools would you recommend for tha t?
If you just want to get data from a db and turn it into html on the serverside (like php) then I'd suggest getting started with jvm clojure / ring
How about this example for a simple DB-backed (server-side) web app? https://github.com/seancorfield/usermanager-example
https://practical.li/clojure/ John has a bunch of docs that I find helpful.
Thanks @U04V70XH6 ! I am going to try this !
And also https://clojure-doc.org/articles/tutorials/basic_web_development/ to walk you through a basic Clojure web app from scratch.
Thanks Ed ! Yeah I have that website in my resources channel for Clojure. It's just that there's so many tools and terms that get thrown around and when I am trying to learn a simple thing, it gets a little confusing on WHAT I should start with since each Youtube video has different tools for doing similar things.
Thanks @U04V70XH6 I'll read through this !
I did and was having so much fun but it kept breaking on me. So I switched to EMACS but then it broke too lol. I am now just using plain Notepad++ and use http://TryClojure.org REPL to test stuff.
I worked with someone once that learned Clojure and Emacs at the same time. I really wouldn't recommend it.
My setup on Windows is VS Code + Calva with all my Clojure stuff on WSL2 (Ubuntu) so I don't have to deal with CMD or Powershell (since none of the Clojure docs out there are written for Windows -- just macOS/Linux)
I used to use Emacs in the early 80s through mid-90s 🙂 And picked it back up when I switched to Clojure in 2010... but I never settled back into Emacs and kept trying other editors until I settled on VS Code...
I will try to fix my VS Code and use it. Yes, CALVA was alot of fun !!! But one day for whatever reason, the REPL wasn't jacking in like it usually did, and I didn't even change anything. Reinstalled and got help for Reddit. I'll revisit VS Code with Calva and see if I can fix it.
Thanks guys for your help ! You can see me more in here from now on xD (sorry for the future headaches you guys will undergo )
It's much easier to work with Clojure/Script (and Calva) if you are prepared to go the WSL2 route -- but I'm not sure how that works on Windows 10 (I've been on Windows 11 for a long time).