Fork me on GitHub
#clojure
<
2021-03-31
>
seancorfield00:03:52

So… you’d either want a way to specify additional items for the compilation classpath, or a way to exclude JAR files (currently the regex-based exclude operates only at the file level, not on the JAR or its contents)…

seancorfield00:03:26

Open an issue on the depstar GH and I’ll have a think about it…

🙌 3
agentfunk00:03:07

@seancorfield will do. Thanks for the great tool and your answer.

seancorfield00:03:42

I suspect a reasonable way to approach it is to provide an option to specify :compile-aliases so you can tell depstar to calculate a different basis for AOT compilation than for the JAR building part. Then you could have a :storm alias that adds that JAR via :extra-deps (and you’d have to use it for developing with). But “additive” is better.

agentfunk01:03:24

Is there a way to share what goes in :deps with :storm and :compile-alias in your example? Say I have 20 libs in :deps that are used in :test and others, except one that I want to be excluded. The additive approach would need to have :deps referring to the union of what is needed in compilation and uberjar. Otherwise, I'd have to duplicate the dependencies, split between two alias that are used by uberjar.

seancorfield01:03:14

My suggestion was: add a :storm alias that has just that storm-client dependency — and remove it from your “normal” list of dependencies. Then you use that alias (along with whatever you normally use) for dev/test — and if I add a :compile-aliases option you would also pass it to that, since it’s needed for the compile part but not the main JAR-build part.

seancorfield01:03:43

At work, we have a somewhat similar situation where we have a process that needs the standard root-level :deps alias (as in tools.deps.alpha) for a couple of actions but we don’t want it in the JAR version of that process, so we just specify -M:deps:other:aliases when we run those particular actions and -M:other:aliases when we run other stuff or build the JAR. That code doesn’t need it statically for compilation, only dynamically to run certain actions. Hence, I haven’t needed the :compile-aliases option at work yet.

seancorfield01:03:37

And to be clear :compile-aliases would be a new exec arg that you pass to depstar — a variant of the existing :aliases exec arg.

agentfunk01:03:35

@seancorfield yeah that would work great

seancorfield02:03:18

@U01RKE3LQ73 A first cut of the implementation can be tested via

seancorfield/depstar {:git/url "" :sha "39d234b9bce94ff8f51e0660f9db8bf475618e23"}
Needs tests and documentation but I figured you might be able to take it for a spin and let me know it works for you? (edited to update the SHA)

agentfunk03:03:17

@seancorfield thank you so much. I’ll report back tomorrow morning

seancorfield04:03:30

@U01RKE3LQ73 FYI -- once I added tests, I found a bug, so I updated the SHA above to reflect the correction.

🙌 3
agentfunk14:03:58

works perfectly!

agentfunk14:03:19

thank you so much

agentfunk15:03:20

I haven't figured out how to make cursive resolve linter dependencies from aliases. But that's ok for storm since it can't resolve a lot of things from storm.

seancorfield16:03:37

I released it as com.github.seancorfield/depstar {:mvn/version "2.0.211"}.

🚀 3
agentfunk01:03:24

Is there a way to share what goes in :deps with :storm and :compile-alias in your example? Say I have 20 libs in :deps that are used in :test and others, except one that I want to be excluded. The additive approach would need to have :deps referring to the union of what is needed in compilation and uberjar. Otherwise, I'd have to duplicate the dependencies, split between two alias that are used by uberjar.

onetom06:03:26

Does anyone know about a generic OAuth client, which supports the PKCE flow (as defined in https://tools.ietf.org/html/rfc7636)? I'm not even sure what should I use as a generic OAuth client...

onetom06:03:54

I forgot to search clojars. There are a few libs there which have oauth in their name. I've also looked into specifically Xero API related libs and I found that the https://bitbucket.org/icm-consulting/clj-xero/src/develop/ project for example is using the https://github.com/eraf2135/clj-oauth fork of https://github.com/mattrepl/clj-oauth Then there is https://github.com/r0man/oauth-clj from the prolific r0man or https://clojars.org/oauth/oauth.two from the very official-sounding oauth clojars org. So my question more precisely is: which one of these libraries have you used and what was your experience like?

emccue13:03:44

@onetom Sometimes I feel like the only clojure dev that uses https://github.com/spring-projects/spring-security

👍 3
onetom16:03:07

thx for the link. i will probably use it to see how they named certain things, but i definitely won't unleash such a monstrous class hierarchy on my colleagues. imean, we are using clojure to avoid things like: OAuth2AccessTokenResponseHttpMessageConverter.setTokenResponseConverter() (i've found this on https://docs.spring.io/spring-security/site/docs/current/reference/html5/#oauth2client)

onetom16:03:43

we have a ~200 LoC implementation, using http-kit, clojure.data.json and a hint of buddy-core for the sake of convenience. it's not super generic and some of the names are a bit quirky, but it's rather straightforward...

mjw17:03:30

I have not even considered that Spring Security could be used effectively from Clojure :thinking_face:

emccue18:03:35

@UGTAV6LR2 Yeah, the one thing I use it for consistently is DelegatingPasswordEncoder

👍 3
vemv19:03:42

is there a compelling reason to use Sping Sec? leaving aside its provenance, depending on a monolithic (?) solution doesn't seem very aligned with clojure ideals.

onetom02:04:01

@U3JH98J4R im curious too! have u been already familiar with spring sec before u learnt clojure? @U45T93RA6 let's not forget that one of clojure's core ideas is to allow leveraging existing jvm solutions.

emccue02:04:56

I'm not going to pretend to have a real scientific justification but in general for stuff like passwords and hashing i'd rather have stuff that is more well audited. Its a comfort blanket sort of thing.

emccue03:04:03

I tend to program in clojure as if everyone involved in the language other than me is going to die suddenly one day choking on the dust that was once a river

emccue03:04:49

JDBC stuff is stable, http stack is stable, libraries that work with just data I can alter if needed

emccue03:04:25

"Security" just feels like something that would require frequent/crucial updates from domain experts. I trust the spring community for that more

emccue03:04:17

Which is hypocritical, since I do use buddy for stuff

vemv03:04:40

IDK, a crypto fn is essentially a mathematical function and I don't expect math to change much over time :) FWIW I'm not java-allergic (in fact the opposite) but if I can pick a lib instead of a framework, that seems a win to me auditing is also a delicate topic. A framework (even more so one written in java) seems hard to audit; often surprisingly few people know the guts of the stuff

vemv03:04:27

it's not all kittens and unicorns in clojure land of course, but a smaller lib in my language of choice seems vastly easier to assess (especially for code coverage)

emccue03:04:30

> a crypto fn is essentially a mathematical function Yeah, but the time taken to compute that function is an important consideration. I remember reading this a while back https://security.stackexchange.com/questions/18197/why-shouldnt-we-roll-our-own and it i think influences my opinion

emccue03:04:04

There is this offhand comment in there "a scheme isn't secure until it has been extensively attacked."

🙂 3
emccue03:04:37

and at this point i'm arguing with vibes and impressions, but it does make sense to me

emccue03:04:28

at least in the manner I use spring security it is a library - it has compile time dependencies on spring but I don't think much at runtime, and I just call the java classes directly without any of that factory/bean nonsense

👍 3
emccue03:04:16

(def ^PasswordEncoder password-encoder
  (PasswordEncoderFactories/createDelegatingPasswordEncoder))

(defn create!
  "Creates and returns a user record."
  [db email password]
  (sql/insert! db "\"user\"" {:email (string/lower-case email)
                              :password_hash (.encode password-encoder password)}))

(defn confirm-password
  "Takes a user and a password to check to see if it is the password for the user."
  [user password]
  (let [password-hash (:user/password_hash user)]
    (.matches password-encoder password password-hash)))

emccue03:04:33

(side project, not work project so I can share code)

vemv03:04:09

I respect all of that :) probably the world would be a safer place if there more lang-independent test suites around for crypto, csrf, etc. Like https://github.com/nst/JSONTestSuite but for this domain Else our security boils down to trust. There isn't a categorical difference between trusting Spring or trusting a high-quality clj lib

emccue03:04:25

Yeah, which - there is a categorical difference between stuff like buddy and https://github.com/jcf/oauth-two

emccue03:04:05

no offense to jcf or his code. I can't think of any obvious things he would get wrong or can even justify oauth being an issue

emccue03:04:48

but it is marked as "This project is under active development, and has yet to reach 1.0. As such the API may change." in the readme, has 4 releases, hasn't been updated in 5 years, and has 17000 downloads on clojars

onetom14:04:37

thanks for the extensive discussion!

aengelberg20:03:09

Is there a way to profile namespace loading in Clojure and figure out which namespaces are taking the longest to load? I have a test suite that takes a minute or two just to load all the requisite namespaces before the assertions start happening, and am curious where that time is going and what code / dependencies I should try to get rid of.

hiredman20:03:31

the ones with the macros

thinking-face 3
hiredman20:03:21

what version of clojure?

hiredman20:03:45

somewhere around 1.10 there was a change to how user.clj is loaded, where loading code from user.clj could take a long time

hiredman20:03:58

(the change fixed it)

hiredman20:03:18

basically if you had a user.clj, and it loaded any core.async code it could take minutes

ghadi21:03:56

1.10.1 was the fix

noisesmith20:03:28

this quick and dirty might be good enough

(cmd)user=> (alter-var-root #'require #(fn [& args] (println "require" args) (time (apply % args))))
#object[user$eval3$fn__4$fn__140 0x320494b6 "user$eval3$fn__4$fn__140@320494b6"]
(cmd)user=> (require 'clojure.set)
require (clojure.set)
"Elapsed time: 4.618158 msecs"
nil
(ins)user=> (require 'clojure.string)
require (clojure.string)
"Elapsed time: 0.074134 msecs"
nil

dpsutton20:03:35

isn't there a dynamic var that does essentially that?

noisesmith20:03:04

IIRC there's a dynamic var to make it verbose, I didn't recall it printing timings and I am failing to recall the name

borkdude20:03:29

clojure.core/*loading-verbosely*

Alex Miller (Clojure team)20:03:49

^^ or you can put :verbose flag in the first require (which sets that)

👍 3
borkdude20:03:10

oh I didn't know that one, thanks

Alex Miller (Clojure team)20:03:55

then you can usually just eyeball it

noisesmith20:03:15

oh - it does print the timing, nice

borkdude20:03:02

what prints the timing?

noisesmith20:03:39

require prints the timing if you set *loading-verbosely*

(ins)user=> (require 'clojure.walk :verbose)
require (clojure.walk :verbose)
"Elapsed time: 0.068989 msecs"
nil

borkdude20:03:56

no, that's just the side effect of your alter var root

😹 6
aengelberg20:03:19

oooh awesome

Alex Miller (Clojure team)20:03:31

but the answer is: clojure.core.async

😂 9
💯 3
ghadi20:03:42

clojure.tools.analyzer 😉

potetm20:03:27

yet another alternative: use clojure.tools.namespace

potetm20:03:45

topo-sort the namespaces then load them one at a time with a timer

😂 3
potetm21:03:00

tbc — not saying issue individual require commands. Do it in a doseq.

potetm21:03:27

But you can just time each require in the standard way once they’re topo-sorted.

richiardiandrea21:03:45

Hi all, I am a bit stuck on this. How do I tell the compiler that I want the first arity of the following Paths/get? https://docs.oracle.com/javase/8/docs/api/java/nio/file/Paths.html I tried: (Paths/get #^"[S" (into-array [(:path image)])) I might have syntax error - I googled a bunch of stuff...

richiardiandrea21:03:35

Sean saved my chickens, thank you! For posterity the correct call is:

(Paths/get (:path image) ^"[Ljava.lang.String;" (into-array String []))

seancorfield21:03:56

@richiardiandrea Only the ... part maps to an array, so you want a String followed by an array of String.

richiardiandrea21:03:25

oh man, that's silly 😄

Alex Miller (Clojure team)21:03:34

#^ is old school now, just ^ is preferred

👍 3
ghadi21:03:17

@richiardiandrea you do not need the type hint

user=> (set! *warn-on-reflection* true)
true
user=> (Paths/get "/usr" (into-array ["local"]))
#object[sun.nio.fs.UnixPath 0x1a411233 "/usr/local"]

richiardiandrea21:03:38

I am basically parsing only one string though, sorry it was not clear maybe from the example

ghadi21:03:54

👌:skin-tone-3: that makes sense

richiardiandrea21:03:34

Still haven't solved though, cause this fails

(Paths/get "/home" ^"[S" (into-array ^String []))

richiardiandrea21:03:13

ah, ok found out - no type hint in into-array

richiardiandrea21:03:33

(Paths/get (:path image) ^"[S" (into-array String [])) works

ghadi21:03:39

[S is not a valid type hint

ghadi21:03:55

should be [Ljava.lang.String;

ghadi21:03:54

also into-array takes a class directly -- not a type hinted arg:

(into-array String [.....])
not
(into-array ^String [.....])

ghadi21:03:19

user=> (-> (make-array String 0) class .getName)
"[Ljava.lang.String;"

hiredman21:03:11

isn't [S an array of shorts?

hiredman21:03:58

user=> (type (make-array Short/TYPE 0))
[S
user=>

ghadi21:03:24

only the primitives have single letter abbrevs

richiardiandrea21:03:45

Completely misread that S here lol https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-4.html#jvms-4.3.2-200 "Oh it's an S - same as the first letter of String" 😅

ghadi21:03:47

(because the signatures of Paths/get have distinct # of args)

borkdude21:03:47

@richiardiandrea if you're interested in a library around java.nio.file which makes this stuff easier from clojure, check out https://babashka.org/fs/babashka.fs.html

richiardiandrea21:03:57

yeah just discovered that thank you

borkdude21:03:56

the array stuff is what bugged me most ;)

richiardiandrea21:03:00

it took me 30 mins + asking here -> I have just replaced everything we mentioned above with babashka.fs functions now 🤷

onetom04:09:48

here is a more recent URL for this lib: https://book.babashka.org/#babashkafs which further points to: https://github.com/babashka/fs

❤️ 1