Fork me on GitHub
#clojure
<
2021-04-21
>
tlonist04:04:28

hello, newbie here 🙂 quick question about developing a restapi server with clojure. I’m currently using http-kit and compojure, and I wonder if there is any clojure-specific conventional structure for api server? Typically with Java (springboot, for example), I’m accustomed to dividing the project into controller, service, and dao. I’ve seen some github projects in clojure that did almost the same; but I wonder if there’s any better way (or clojure way) of doing so.

seancorfield04:04:54

@tlonist.sang The only real guidance in Clojure is to try to keep side-effecting code from pure code but folks organize their web apps in all sorts of different ways, depending on which approach to design they follow.

seancorfield04:04:18

You won't find things like DAOs really because we have no objects 🙂 but for basic CRUD most folks will just traffic in hash maps going in and out of the DB via next.jdbc (which supersedes clojure.java.jdbc).

seancorfield04:04:59

Separating your pure business logic out from both the HTTP side of the house and the persistence side of the house is fairly common.

tlonist04:04:41

@seancorfield Wow, from the author himself! I’ve been using next.jdbc throughout my project this year. Thanks for the guidance. I like how there is no set-rule!

seancorfield04:04:45

If you've read Clojure Applied, I suspect that's going to be some of the best "guidance" you'll find. Perhaps a few of the Domain-Driven Development talks around Clojure (I haven't really followed that path but some people like it a lot).

👍 3
🎉 3
heyarne05:04:33

is anyone using deps.edn on nix(os) and got jogl to work? it seems i can't for the life of it figure it out and could very much use any help 🙂

Timofey Sitnikov10:04:06

Good morning Clojurians, can anyone point me to a good way to handle passwords in an edn file? I do not want to push passwords into repo, is there a common way to handle it? For example in configuration like this:

:postal/email-config {:host  ""
                        :email ""
                        :pass "password"
                        :port 587
                        :tls true}}

👀 3
flowthing11:04:11

Aero handles this kind of thing quite well, I think: https://github.com/juxt/aero#hide-passwords-in-local-private-files If you don't (want to) use Aero, you could just add a tagged literal (https://clojure.org/reference/reader#tagged_literals) that does something similar, I guess.

3
3
p-himik11:04:04

For GMail specifically, you should use an application token instead of the full email account credentials.

p-himik11:04:56

On that particular section of Aero documentation, see some discussion here: https://github.com/juxt/aero/issues/73

3
p-himik11:04:55

FWIW, Heroku uses this approach: https://12factor.net/config Some more in-depth description of the Heroku approach specifically: https://blog.heroku.com/twelve-factor-apps

Timofey Sitnikov11:04:00

@U2FRKM4TW, assuming that the machine is secure and I am the only user of the machine, being able to read the env data is not a concern?

p-himik11:04:40

When a machine is indeed secure, it is not a concern. When a machine is not secure, everything is a concern - you simply cannot store sensitive information there, at all.

Timofey Sitnikov11:04:32

OK, this makes sense.

Yehonathan Sharvit11:04:38

I am looking for a way to check whether two huge maps (around 600K entries) are equal. The issue is that some of the values are of type org.joda.time.DateTime and the values are not always equal, although they refer to the same time. I would like to find an implementation of a function like clojure.data/diff that allows to customize the definition of equality. I could use clojure.data/diff but the performance is not good enough for my huge maps.

andy.fingerhut11:04:02

One possibility would be to replace all occurrences of org.joda.time.DateTime in the two maps you want to compare with a new type of value that does return true for clojure.core/= when you compare them, and then compare those maps using clojure.core/=

andy.fingerhut11:04:20

or clojure.data/diff

andy.fingerhut11:04:02

Or if there is some kind of method you can call on instances of org.joda.time.DateTime that returns a "canonical" value for each instance, such that the canonical values are clojure.core/= to each other when they are the same time, then you don't need a different type.

andy.fingerhut11:04:17

An example of the "wrapper type" approach would be to create a custom Clojure deftype, and define your own custom equiv method (which is what clojure.core/=calls under the JVM hood) that calls the isEquals method of joda.time instead of equals: https://www.joda.org/joda-time/apidocs/org/joda/time/base/AbstractInstant.html#isEqual-org.joda.time.ReadableInstant-

Yehonathan Sharvit12:04:19

@U0CMVHBL2 Is equiv a method of a protocol in Clojure?

Yehonathan Sharvit12:04:34

I know that in ClojureScript it is

andy.fingerhut12:04:14

It is not in Clojure, no. ClojureScript was implemented years after Clojure, and some have asked whether going back to reimplement Clojure using protocols, similar to how ClojureScript does, would be useful, but it would be a big churn in the code base for probably not many advantages, and would break any libraries that rely on the Java class structure in Clojure's implementation today.

andy.fingerhut12:04:01

Note: I haven't actually tried writing the deftype that defines its own equiv myself, and could be forgetting some nuances, but I can point you at existing libraries that do this successfully, e.g. priority-map https://github.com/clojure/data.priority-map/blob/master/src/main/clojure/clojure/data/priority_map.clj#L325

andy.fingerhut13:04:54

Hmmm, looking at the definition of clojure.lang.Utils/equiv again to remind myself, it looks like overriding equiv will work if your type implements the IPersistentCollection interface, but otherwise your custom type would need to override equals

andy.fingerhut18:04:53

Why a bummer? The same approach of defining your own deftype as a wrapper object for times, with your own custom definition of equals, should work, I think.

andy.fingerhut18:04:32

I'm just clarifying that the method you need to override is not equiv, but equals. clojure.core/= calls equiv on some objects, typically ones defined in Clojure itself, but falls back to Java equals for all others.

Yehonathan Sharvit18:04:36

Is there a way to re-implment equals for DateTime or is it sealed?

andy.fingerhut22:04:58

This page of Java doc says that the class org.joda.time.DateTime is final, meaning no subclassing in Java: https://www.joda.org/joda-time/apidocs/org/joda/time/DateTime.html?is-external=true

andy.fingerhut22:04:55

And there is no way to override a method for an existing class in Java that I know of (others might). Wrapping the class in another type and delegating all methods but equals is always possible regardless of final or not.

marciol18:04:27

@U3Y18N0UC and I had this same problem recently

mauricio.szabo19:04:14

Yes, the way I ended up doing was to implement a deep-compare, above

mauricio.szabo19:04:48

You can add an (instance?...) check on the cond above, and add the data comparision

mauricio.szabo19:04:54

(the map that I return is mostly because I used it on a test, so this would become the custom message so I could know, when things failed, what's wrong. Not really that useful when I had to sort collections, to be honest, but it helped compare two 3.5mb-sized data structures)

Yehonathan Sharvit11:04:13

Maybe someone knows a lib out there that provides such a function. I’d prefer avoiding to write an implementation of my own, if possible

Marc O'Morain12:04:45

A question about reading Clojure docstrings:

(dotimes bindings & body)

bindings => name n
 Repeatedly executes body (presumably for side-effects) with name
bound to integers from 0 through n-1.
Where is the form of the bindings documented? Is there anywhere that says that bindings should be a vector?

Marc O'Morain12:04:04

I always have to look on http://clojuredocs.org to see an example usage for functions like this, I find the dosctring doesn’t give me enough explanation to know how to write the form.

thheller12:04:50

don't know where it is documented but bindings are vectors yes

nilern12:04:02

For dotimes, it just isn't documented

nilern12:04:47

> (doc let)
-------------------------
clojure.core/let
  (let [bindings*] exprs*)
([bindings & body])
Special Form
  binding => binding-form init-expr

  Evaluates the exprs in a lexical context in which the symbols in
  the binding-forms are bound to their respective init-exprs or parts
  therein.

  Please see 
Spec
  args: (cat :bindings :clojure.core.specs.alpha/bindings :body (* any?))
  ret: any?

nilern12:04:57

More common stuff like let has Specs for this purpose

restenb12:04:19

is there some built-in utility to "parse" error traces of type clojure.lang.ExceptionInfo ?

vlaaad13:04:57

there is also Throwable->map to convert exception to data

👀 3
🙏 3
vlaaad13:04:34

and clojure.main/ex-triage to analyze that exception

restenb13:04:04

not sure it's necessarily just a normal exception. clj-http generates this whenever a request bounces, for example: clojure.lang.ExceptionInfo: clj-http: status 401 {RESPONSE BODY HERE}

restenb13:04:18

i'd just like to parse the response body directly

restenb13:04:51

perhaps it's just (.getMessage ex) then

restenb13:04:10

at least that should give me status 401 or the body

nilern13:04:47

Hopefully ex-data gives you the body

nilern13:04:00

I thought you wanted to traverse the stack trace

nilern13:04:29

clj-http seems to use Slingshot, so I don't know whether ex-data works like I would like but yeah at least ex-message/`.getMessage` gets you something

borkdude14:04:01

In a Clojure project, it doesn't matter if I AOT with Java 11 or Java 8, because the Clojure compiler only ever emits Java 8 compatible bytecode, is this correct? I understand it matters for Java classes but not for compilation of Clojure units, true?

jumar15:04:58

I think it may matter in rare cases like ByteBuffer.flip() method - I saw errors like this when an uberjar was compiled with Java 11 but run with Java 8: https://stackoverflow.com/questions/61267495/exception-in-thread-main-java-lang-nosuchmethoderror-java-nio-bytebuffer-flip

jumar15:04:41

I guess there's no workaround for that or? (javac seems to have --release parameter for cases like this)

jumar15:04:16

Ah, I didn't notice the rest of the conversation in the main channel - so I suppose that this can become a problem and there's no workaround other than using an older java when building the uberjar

dpsutton14:04:29

the reflector might choose methods that exist in java 11 that do not exist in java 8 if you build in 11 and then run in 8 right?

borkdude14:04:33

True, but then you're programming against JDK 11 specific methods anyway, so AOT isn't the culprit

dpsutton14:04:16

i thought there was some "first acceptable method" type thing to the reflector as well. Doesn't that mean it could emit a call to something that won't exist in java 8? Even though you aren't explicitly using methods from java 11?

dpsutton14:04:22

(I'm way out of my depth here so perhaps I'm stretching, but i bet the nature of any issues that pop up will be quite annoying to diagnose, so perhaps stretching is called for)

nilern14:04:27

At least theoretically they could add some overload Object get(Object) when previously there was only Object get(long) and now you are linked to the new one without reflection whereas if you AOT in Java 8 you get a reflective call

dpsutton14:04:16

i suspect ghadi or alex would have some experience in this if they see this conversation

nilern14:04:48

javac can mess this up too: > Note: Merely setting the target option does not guarantee that your code actually runs on a JRE with the specified version. The pitfall is unintended usage of APIs that only exist in later JREs which would make your code fail at runtime with a linkage error. To avoid this issue, you can either configure the compiler's boot classpath to match the target JRE or use the http://www.mojohaus.org/animal-sniffer/animal-sniffer-maven-plugin/ to verify your code doesn't use unintended APIs. In the same way, setting the source option does not guarantee that your code actually compiles on a JDK with the specified version. To compile your code with a specific JDK version, different than the one used to launch Maven, refer to the https://maven.apache.org/plugins/maven-compiler-plugin/examples/compile-using-different-jdk.html example. > https://maven.apache.org/plugins/maven-compiler-plugin/examples/set-compiler-source-and-target.html

borkdude14:04:00

@nilern in javac you can set --release

Alex Miller (Clojure team)14:04:08

from JDK perspective, there is some danger of resolution using newer JDK methods if they are available in the JDK you are using

Alex Miller (Clojure team)14:04:41

this happened with Clojure in fact when they added a new Collection method override in Java 10 (11? don't remember) that made existing Collection impls ambiguous

borkdude14:04:49

so tl;dr it's probably always better to use a jdk 8 if you target jdk 8 in your CI?

Alex Miller (Clojure team)14:04:26

if you want it to be compatible with Java 8+ that is probably safest

Alex Miller (Clojure team)14:04:09

you can actually tell javac to use a different version of the jdk, I guess you could run the Clojure compiler with a jvm using a bootclasspath set to an older jdk possibly

borkdude14:04:26

do you mean --release ?

Alex Miller (Clojure team)14:04:43

I don't know what the modern affordance is, maybe that's it

borkdude14:04:48

but the Clojure compiler doesn't need / use javac right?

Alex Miller (Clojure team)15:04:04

was just making an analogy

borkdude15:04:33

thanks y'all

Alex Miller (Clojure team)15:04:05

--release is for javac?

nilern15:04:30

Yes it is a javac option

Alex Miller (Clojure team)15:04:20

I guess replaces the old source/target stuff

dazld15:04:28

somewhat related, and realise this is an open question, but what kind of JVM settings do people apply when running clojure code specifically - thinking of things like tuning GC, string compression..? are there any settings that are more relevant for clojure code compared to straight java..?

seancorfield15:04:15

@dazld I think that’s going to be very dependent on the particular application you’re running.

seancorfield15:04:58

Are you going to tune for response times, throughput, fast startup? etc.

dazld15:04:17

I suppose I had at the back of mind that clojure code might not look like typical java code, when it's been compiled, and as such the default settings might not be quite right. If that's not true, then that answers the question.

Alex Miller (Clojure team)15:04:00

Clojure is designed so that the default settings should be sufficient

Alex Miller (Clojure team)15:04:31

if you're running an app in production, you should probably think harder about which GC you're using and how it's tuned

👍 3
Alex Miller (Clojure team)15:04:40

but that will be application specific

ghadi16:04:13

gc throughput -> parallel collector gc latency -> G1 or ZGC speaking broadly

☝️ 3
ghadi16:04:04

in terms of HotSpot/compiler - leave the defaults

seancorfield16:04:41

@dazld As a follow-up, specifically on Ghadi’s response, at work we’re on JDK 11 in production and using the G1 collector and that works great for interactive apps and services where latency is important.

dazld16:04:51

@alexmiller @seancorfield @ghadi thanks a ton, that was all really helpful

nilern16:04:40

I've only used the parallel collector for batch imports and such

ivana16:04:05

Hello! Which symbols can be a word separators in Clojure(script) exept of all the types of brackets and spaces?

flowthing16:04:04

You can probably glean the answer from this page: https://clojure.org/reference/reader

3
Alex Miller (Clojure team)16:04:16

note that there are things that are explicitly allowed, things that are not allowed, and things left intentionally ambiguous for possible future expansion

ivana16:04:05

Editors default is ~!@#$%^&*()-=+[{]}\|;:'",.<>/?` but seems have to be excluded most of them

Janne Sauvala18:04:41

I’m watching Nada Amin’s “Programming should eat itself” Strange Loop talk and the concept of meta levels was interesting. Has anyone built something similar on top of Clojure?

Alex Miller (Clojure team)18:04:07

you can start nested clojure.main/repl's from inside the Clojure repl - they can be set up to support exiting the nested repl, so you can get some small part of this. what she demonstrated is probably beyond that though

Janne Sauvala18:04:17

Cool! I was just thinking how to launch nested repls:ok_hand::skin-tone-2:

flowthing18:04:26

Note that nREPL currently doesn’t support nested REPLs, though. Need to use socket REPL or prepl.

Janne Sauvala18:04:28

Right, I’ll start playing with those repls then

hiredman18:04:28

nesting clojure.main/repl's is also a neat way to support remote repls. a nested repl takes over the io streams until it exits, so for remote repls you do the same thing, but instead of taking over the io streams and running a repl, you take over the io streams and wire them to a repl running somewhere else

flowthing18:04:17

I'm kinda interested in everything related to nested REPLs. When would you do something like that instead of just starting a socket REPL in the remote process and connecting to that?

vemv19:04:48

haven't watched the talk but https://github.com/gfredericks/debug-repl might be considered a nested one?

hiredman19:04:07

https://gist.github.com/hiredman/1789849d21be38310694dbf214d60d34 is a poc for interrupting prepl evaluations, but it shows kind of half of the remote repl thing, connecting and sending things, but not wiring up the current repls io to the remote repl

👀 3
hiredman19:04:53

https://gist.github.com/hiredman/a2630ea6153d06840a2723d5b2c9698c let's you open a repl at a point in your program and "send it" to connect to your existing repl

dpsutton19:04:16

i made a history saving repl recently. and i want to add support to saving history and annotating each subform with it's evaled result, similar to the CIDER debugger

dpsutton19:04:22

that's awesome!

borkdude19:04:50

it even works with bb :) (that's how I discovered it, someone told me he used it with it)