Fork me on GitHub
#clojure
<
2020-03-13
>
yuhan01:03:35

Does this definition make sense? Trying to make a wrapper similar to priority-map-keyfn without falling into pitfalls with custom comparators

(defn sorted-set-keyfn
  [keyfn & keys]
  (apply sorted-set-by
    (fn [a b]
      (let [c (compare (keyfn a) (keyfn b))]
        (if (zero? c)
          ;; break the tie consistently
          (compare (hash a) (hash b))
          c)))
    keys))

leonoel09:03:20

(sorted-set-keyfn count "Aa" "BB")
#_=> #{"Aa"}

leonoel09:03:00

sorted-set and sorted-map require total ordering because the comparator is also used for equality. hash can't be used to break ties because of collisions.

leonoel09:03:23

I don't know if there's any practical reason why partial ordering could not be supported though

yuhan12:03:13

huh, that's interesting - I had assumed hash collisions would be much harder to come by

rickmoynihan12:03:04

Regarding this: https://clojure.org/guides/dev_startup_time I recently tried using clj-kondo metadata to find the subtree’s of ns dependencies that are referenced by my project, but I get weird errors when running with the classes directory on my cp. Stuff like ClassNotFoundException’s. One thing to note: the project uses ig/load-namespaces

alexmiller13:03:53

I don't know what that is

alexmiller13:03:06

if you are partially compiling things and missing things you depend on, it's possible to get into bad scenarios. what happens if you do what's in the guide?

rickmoynihan13:03:27

I can’t literally do what is in the guide; because the app uses ig/load-namespaces; essentially the require tree is built dynamically by wiring in integrant config… esssentially stuart sierra’s dependency.

vlaaad13:03:57

ig is for integrant

👍 1
vlaaad13:03:45

can integrant be a red herring?

rickmoynihan13:03:39

I’m not suggesting it is a herring; just that it and also perhaps the presence of tools.namespaces.repl/clear etc might be 🙂

vlaaad13:03:16

load-namespaces just ends up calling require — same as ns macro ends up calling require

rickmoynihan13:03:07

true — it’s really the use of integrant.repl/reset which causes me concern; rather than load-namespaces.

rickmoynihan13:03:25

as that uses clojure.tools.namespace.repl/refresh

rickmoynihan13:03:05

though I suspect it’s probably me messing up in my suggestions of namespaces to compile

rickmoynihan13:03:49

@alexmiller Yeah ok it makes total sense that compile assumes all transitive requires are also compiled. It’s most likely in my manual pass over the first results that I messed something up regarding transitive namespaces… I had to for example remove some shadow-cljs npm requires; and also some cljs namespaces etc by hand; as it’s a mixed clj/cljs project.

reborg13:03:44

It remembers me of an old https://groups.google.com/d/msg/clojure/NtQ9BJXbziA/AlONiMe9I0AJ discussion related to calling lein uberjar (which adds compiled classes to the classpath) and then starting a REPL. Same class not found error and same relationship with having a component system in place. Never spent time to find the root cause, but now I always lein clean after uberjarring.

rickmoynihan14:03:46

I might be misunderstanding the issue, but I think these problems are caused by :aot poluting other profiles. I always prevented that with: https://github.com/technomancy/leiningen/blob/0455879f9a42629205e9cedb4d61f13054a5e1d7/sample.project.clj#L313-L317 No idea why lein never made this a default option tbh; I’ve always set it.

reborg14:03:03

Ah good to know. Will try soon

👍 1
reborg14:03:38

Yeah that did it, good to know :thumbsup:

dominicm13:03:27

There's a lot of fear around aot. Tools namespace explicitly discourages it.

p-himik13:03:49

Random ClassNotFoundException was the exact reason I stopped using AOT for third-party libraries.

alexmiller14:03:21

there is a lot of irrational fear around aot

rickmoynihan14:03:52

Yeah I’ve always AOT’d apps, never really had any issues with it; that weren’t issues worth fixing anyway… e.g. side effects happening at compile/load time.

rickmoynihan14:03:51

Integrant with dynamic namespaces/components makes it somewhat more difficult though — I’ve been wanting to resolve that for a while.

rickmoynihan14:03:36

One thing the startup time link above doesn’t discuss is the pitfalls of doing it… e.g. presumably an app that downgraded a dependency to a prior version might have some problems

alexmiller14:03:04

"Periodically, you will need to re-compile to account for new dependencies or changing code."

👍 1
rickmoynihan14:03:11

> AOT-compilation: Reloading code does not work in the presence of AOT-compiled namespaces. If you are using AOT-compilation in your project, make sure it is disabled and you have deleted any AOT-compiled .class files before starting a REPL development session. (In Leiningen, run lein clean.) Another example of a potential gotcha would be the above, though is the statement from https://github.com/clojure/tools.namespace/ even accurate though? I suspect it’s more nuanced than it lets on. i.e. presumably reloading will work in the presence of AOT-compiled namespaces; so long as the namespace that is being reloaded isn’t itself AOT-compiled? Is that right?!

alexmiller14:03:47

without running experiments, don't know

👍 1
adamfrey14:03:29

is there a way to use namespace aliases to creating a record that is defined in another ns?

(ns app.worker)

(defrecord WorkerA [x] ...)

(pr-str (map->WorkerA {:x 1})) ;; => #app.worker.WorkerA{:x 1}

(ns app.other
  (:require [app.worker :as w]))

#app.worker.WorkerA{:x 1} ;; works

#::w.WorkerA{:x 1} ;; doesn't work

alexmiller14:03:35

you should really prefer using the constructor functions, which are normal functions and can be accessed via aliasing in the same way as any other functions

alexmiller14:03:13

so here (w/map->WorkerA {:x 1}) would work fine

adamfrey14:03:36

right. Just checking for completeness. Thanks, Alex

alexmiller14:03:11

the #app.worker.WorkerA is a syntax for a Java class name constructor and Java class names have nothing to do with namespace aliasing

didibus15:03:23

Definitely I've experienced AOT issues. AOT on your app is fine, uberjar style. But for libraries it's caused us issues in the past for sure

didibus15:03:45

Mostly because compile is transitive

didibus15:03:01

So what happens is you end up if dependency A uses B v1, and you depend directly on A and B v2. Then the AOT jar of A will actually contain the compiled classes of B v1

alexmiller15:03:12

that does not seem right. by what process would that happen?

Derek Passen15:03:45

that’s an AOT uberjar, no?

didibus15:03:12

No it's not. That happens just if you compile A and jar the classes-dir folder

alexmiller15:03:50

that seems like a bad process

alexmiller15:03:05

not blaming you or anything

didibus15:03:59

Ya, it is. What I did at work was we now have a custom jarring. So it'll filter through the classes-dir and select only classes of the package to Jar.

p-himik16:03:23

> That happens just if you compile A Would the same happen if you run compile with the same classpath that your app has?

didibus16:03:33

No, it would be fine then

didibus16:03:58

But only if you had A as a source deps

didibus16:03:34

The issue is, libraries publishing their Jars should not be AOT compiled, or if they are, should be very careful about what compiled classes they put in the Jar they vend

p-himik16:03:50

Hmm. Still at a loss then for why I was seeing ClassNotFoundException from time to time when I was trying to compile the whole all.

didibus16:03:31

Hum.. in the REPL?

didibus16:03:00

Part of that error would depend on how you launch the app

didibus16:03:33

If it's started where the classpath to the JVM doesn't point to all the needed dependencies this would happen

p-himik16:03:43

Not in the REPL, no. The classpath is exactly the same.

didibus16:03:40

Exactly the same?

didibus16:03:27

If you expect to use a generated class from gen-class for example, the classpath post compile must now include the compiled classes

didibus16:03:04

Or it won't find them

didibus16:03:35

But Clojure requires shouldn't be broken, even if they don't find the classes they fallback on the source

didibus17:03:00

import might, like I said. As the classpath must point to the compiled classes

p-himik17:03:52

Well, yeah, I think the only difference was the "classes" dir. Apart from that, the classpath was exactly the same. Most of the time, it wasn't able to find manifold.deferred.IDeferred which is a definterface.

didibus17:03:33

Those kind of errors I feel are always super mysterious

Santiago15:03:45

I’m making a form to upload a file. Simple stuff, only a field for a name, and a field for uploading. I already have some code in place to do this:

[:input {:type "file" :name "uploaded-file" :required ""
                    :accept ".xls, .xlsx, .ods, .csv, .tsv, .pages,
                             application/vnd.openxmlformats-officedocument.spreadsheetml.sheet,
                             application/vnd.ms-excel, text/csv,
                             application/vnd.oasis.opendocument.spreadsheet"}]
but if someone uploads a file through an android phone, they can still upload e.g. a pdf. Am I doing something wrong in that accept attribute? If not, is there a way to do it server-side without adding cljs to my project?

didibus16:03:52

You can't use that as validation

didibus16:03:03

This is part of the HTML 5 spec

didibus16:03:19

Which means it's a matter of each browser supporting it or not

didibus16:03:37

You could always find a browser that doesn't support it

didibus16:03:32

I think it can also be really finicky to find the correct combination of attributes for different browser and OS as well

Santiago16:03:45

ah makes sense.. I’m new to this 😄 cool, so I actually need some JS to do this

didibus16:03:12

Well... To be honest. That part I'm not so sure either

didibus16:03:25

The file picker is not normally something you have direct control over

didibus16:03:44

I'm not sure if JS will be able to configure it anymore

didibus16:03:23

But you will be able say, after the user picks a wrong file extension in the file picker, to do a quick client side validation before the file is uploaded (I think)

didibus16:03:58

Otherwise you can do the same on the server side, though that means the file has to be fully transferred and all that

didibus16:03:20

It can probably help you at this point to Google for HTML file accept or like how to restrict file selection of input file, etc.

didibus16:03:28

Nothing should be Clojure specific

didibus16:03:57

This one seems to have good info

didibus15:03:14

In the case where B is not vending itself as AOT

didibus15:03:24

So because B is not compiled

didibus15:03:34

The compile of A will transitively include all of B

didibus15:03:47

And all those classes will go on the classes-dir

didibus15:03:00

And then get Jared as a lib (not uberjar)

didibus15:03:48

And then when you depend on A, you get the classes of B as well. And if A depends on a different B, and because B is a source lib, and Clojure load favours classes over sources you will get the version of B defined by A

alexmiller15:03:41

yeah, that seems like a problematic way to do aot, but I see how you get there

alexmiller15:03:22

aot'ing from an app namespace makes you dependent on the classpath, which has already been resolved with version selection (only B v2 is available)

alexmiller15:03:52

seems like if you are running all your compile's in that classpath context, same would be true right?

didibus15:03:14

Yes, but A specifies a dependency on B v1

alexmiller15:03:20

what tooling were you using for this or was it something custom?

didibus15:03:34

So the classpath resolves to B v1. And then A is AOT compiled

didibus15:03:49

And then my App depends on A and B v2

alexmiller15:03:56

but the classpath will not resolve to B v1 here

didibus15:03:17

It won't, but the Jar of A will have classes of B v1 in it

didibus15:03:01

And Clojure Load will see that each namespace on B v2 (whose Jar only has sources) also have a .class on the classpath coming from A's Jar

alexmiller15:03:07

maybe I was misunderstanding what you were talking about then. are you talking about publishing aot'ed library artifacts? or a process for aot'ing an app?

didibus15:03:38

I'm talking about publishing AOT libs

alexmiller15:03:42

ah, don't do that

alexmiller15:03:56

and if you do, they should only include that lib's classes

didibus15:03:00

Yeah this is with custom tooling. But I remember looking what lein does and it seemed it would suffer the same faith

alexmiller15:03:33

yeah, it is very easy to do it wrong (and it takes away choices from a downstream app builder)

alexmiller15:03:49

compiled classes are inherently dependent on some version of the clojure compiler

alexmiller15:03:04

if the lib compiles, the app doesn't have the opportunity to pick that

didibus15:03:48

Ya, that's what I meant by libraries shouldn't be AOTed.

didibus15:03:05

Application can and I've never had issues there

alexmiller15:03:06

ok, then we are in violent agreement :)

👍 1
alexmiller15:03:17

sorry I misunderstood :)

didibus15:03:06

No worries, honestly it hurts my brain everytime I try and rethink about this issue. Dependencies are so hard

didibus15:03:57

For us, we had a case of a dependency doing AOT because it also gen-classes itself so it can be consumed by Java and Clojure

didibus15:03:49

So that was even more fun to deal with! 😅

alexmiller15:03:59

yeah, that's hard

alexmiller16:03:36

a couple of the contrib libs actually publish both source and aot versions (the latter under aot classifier)

alexmiller16:03:22

well, actually clojure itself too has a "slim" version that is source-only

didibus16:03:02

I did face this issue with Clojure actually. When going from 1.9 to 1.10 I think. We had a library that was doing an AOT for a gen-class. Even though it was only compiling the gen-class namespace and that had no requires. Transitively, it would compile the only sources not AOT compiled in Clojure 1.9, the core specs.

didibus16:03:58

And somehow on 1.10 some core macro changed in a way that the 1.9 spec would fail to validate calls to it

didibus16:03:25

So we were getting the 1.9 specs of Clojure indirectly through this.

didibus16:03:31

Was very confusing haha

didibus16:03:30

Anyway, our solution now is our build have you list out the namespace you want to gen-class explicitly. And it will do an AOT of them, and then select only the classes file of that namespace and not of any transitive dependencies. And only include that in the Jar

alexmiller16:03:16

that's how basically all aot tooling ends up working

alexmiller16:03:56

that's what we do with the maven clojure plugin for the contrib libs

didibus16:03:42

Ya, I feel I had checked what lein did and it didn't appear to do that. Like it does only AOT the gen-class namespaces. But then it packages everything resulting from that into the Jar

alexmiller16:03:06

yeah, I think lein is really targeted more towards the app casee

alexmiller16:03:30

but it's been a while since I looked at it

didibus16:03:44

I should go see what the maven contrib Clojure plugin does. Might be some good learnings in there

alexmiller16:03:17

the config is pretty fiddly to make it do exactly what you want

alexmiller16:03:07

https://github.com/clojure/tools.reader/blob/master/pom.xml is the tools.reader config which is fairly complicated as it's both building a source jar, then aot'ing, and building an additional aot'ed jar with the assembly plugin

didibus16:03:11

I do think I'm seeing some of the code that cleans up the classes that weren't for the declared namespaces, seems this does it here: https://github.com/talios/clojure-maven-plugin/blob/develop/src/main/java/com/theoryinpractise/clojure/ClojureCompilerMojo.java#L64

marrs18:03:09

Hello, everyone. If anyone here uses vim, can you tell if it's normal for clojure to have different rules about how words are defined? Currently, for me, w pretty much behaves the way I'd expect W to work, and * and # match along similar definitions? I've had some plugins installed from when I tried clojure years ago and I don't know if I've missed one as I've progessively uninstalled everything Clojure or Lisp related

andy.fingerhut18:03:44

I do not know the answer. I suspect you may get some useful replies here, but wanted to point out that in case not, there is also a #vim channel that may be more likely to reach knowledgeable folks in that area.

marrs18:03:20

Many thanks, I'll ask there

orestis19:03:28

Did clojure.data.xml not get the 1.0.0 treatment?

orestis19:03:37

Should still be considered production ready, right? What with one stable and one alpha version. BTW the fact that I’m asking is a good indicator that the 1.0.0 move was a right one :)

alexmiller20:03:56

I would use the alpha right now

huthayfa21:03:13

Hello, Does anyone know how to add and configure gatling-maven-plugin in project.clj https://mvnrepository.com/artifact/io.gatling/gatling-maven-plugin/3.0.5

noisesmith21:03:01

lein doesn't use maven, and I wouldn't assume a maven plugin would be useful in a lein project

noisesmith21:03:12

that said you could make a pom.xml (or even run lein pom to make a project pom) and configure it there

noisesmith21:03:44

this would mean using maven in parallel with lein (or instead of it)

huthayfa21:03:28

if lein does not use maven, why is the plugin available as lein deps

noisesmith21:03:55

the plugin itself is a maven artifact, lein knows how to resolve and download the artifacts (using the underlying aether lib), but it doesn't use the maven executable which is what you need for maven plugins

noisesmith21:03:54

lein uses the same artifact cache / download setup as maven but doesn't run maven plugins

dev.4openID21:03:58

Hi all,curl -X GET ' '"https://reqres.in/api/users?page=2' -H 'API-Key:MySecret'

dev.4openID21:03:21

that was not supposed to happen dmanit

1
bfabry21:03:44

oooooh that string doesn't look like it should be on a public slack channel 😛

dev.4openID21:03:06

yes - it fortunately only a test one

dev.4openID21:03:57

hopefuly it is gone now?

dev.4openID21:03:06

what a mistake!!!! GRRRR

dev.4openID21:03:28

when I use this on a cli curl -X GET '"https://reqres.in/api/users?page=2' -H 'API-Key:MySecret' I get the correct results in clojure snippet ((client/get "https://reqres.in/api/users?page=2" {:body-encoding "UTF-8" :accept :json :jwt ["API-Key" "MySecret"]})) it fails now it might be the auth-type is a jwt but I cannot tell - I have tried all variations of :auth-type "oauth1" :auth and similar and cannot get it right what am I doing wrong? I have been all over the clj-http github docs but ....

noisesmith21:03:34

why are you putting what should be a header into a :jwt key?

Nir Rubinstein21:03:57

:headers {"API-KEY" "SECRET"}

1
dev.4openID21:03:32

:headers {"API-KEY" "SECRET"} results in failuer Key must be integer

dev.4openID21:03:45

hold typo? on my side?

noisesmith21:03:07

"key must be integer" means that some clojure function is doing a map lookup but you gave it a vector

tjb21:03:14

hello everyone. can someone direct me to whom is in charge of keeping the ubuntu leiningen package up to date?

chrisulloa21:03:39

2.9.1-2 is listed

tjb21:03:27

i tried sudo apt install leiningen-clojure but no dice...maybe im using the wrong command tho

tjb21:03:56

and sudo apt install leiningen gives me 2.8.1

chrisulloa21:03:45

have you updated? sudo apt-get update

tjb21:03:33

correct i have

tjb21:03:20

$ sudo apt-get install leiningen
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  leiningen
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 13.0 MB of archives.
After this operation, 14.9 MB of additional disk space will be used.
Get:1  bionic/universe amd64 leiningen all 2.8.1-6 [13.0 MB]

seancorfield21:03:11

Does lein upgrade not work after installing from apt?

tjb21:03:13

correct it says that upgrade is intended to be used outside of a package manager install

tjb21:03:21

which is a bummer

seancorfield21:03:37

Ah, then I'd go with noisesmith's suggestion of just installing it manually.

tjb21:03:21

is showing 2.8.1

tjb21:03:39

but i would like 2.9.1

tjb21:03:42

will bionic forever be on 2.8.1?

noisesmith21:03:47

lein is very easy to manage outside the ubuntu package management system: you put a shell script in ~/bin/lein and then use lein upgrade X to make it run a specific version

2
✔️ 1
tjb21:03:25

ok i will try that then!

noisesmith21:03:56

it's just the one file (and of course making sure that file's directory is on path), as described on that page

tjb21:03:52

so this file is lein and it does not install on the machine?

noisesmith21:03:34

the file is a wrapper script that uses a local cache for holding the actual jar and launching a java process

noisesmith21:03:45

it's both the script and the downloader, in one

tjb21:03:20

interesting.

tjb21:03:10

does it have to be on the path? i assume if i ran the file like ./bin/lein it would install it on the machine, no?

noisesmith21:03:48

no, it does no installation of the script itself, of course it can be run via absolute path but it doesn't put itself on path

tjb22:03:04

hmmm ok sorry im getting confused by like

Run it (lein) and it will download the self-install package

tjb22:03:12

^ makes me assume it will download lein on my machine

noisesmith22:03:45

it downloads a jar that it uses

tjb22:03:48

ok cool i think im understanding

tjb22:03:54

thanks @U051SS2EU for your patience w me!

dev.4openID21:03:46

Yes it works thanks - the docs stated [`"API-KEY" "SECRET"] not {"API-KEY" "SECRET"} many thanks`

dev.4openID21:03:45

but it results in a 401 Unathorised

noisesmith21:03:42

in your original you had API-Key - is it picky about case?

dev.4openID21:03:26

that is my bad typing

dev.4openID21:03:35

So no it is not the case

dev.4openID21:03:07

I have it consistent in the actual test item

dev.4openID21:03:24

I have tried the :digest-auth [api secret]/ the aws version/ etc.... so it is very frustrating

noisesmith21:03:09

if the curl you showed works none of that is even relevant

dev.4openID21:03:28

that is what I find odd

dev.4openID22:03:18

I have also used httpie and got a result

dev.4openID22:03:04

Other docs on a sister site stated they use jwt but as you state - the keys are in the header

ghadi22:03:07

if you really want quality help, and to value people's (free) time, - Post what the intent was - Post the code you ran - Post the error you received (in full, don't leave out things you don't think consider helpful)

ghadi22:03:36

like be really specific, rather than saying stuff like "it failed" or "it worked"

ghadi22:03:55

otherwise it's the Miss Cleo school of debugging

😂 2
dev.4openID22:03:50

thanks, at 9:30 I hope i did state the problem - it is one line of clojure where I am attempting to covert a curl call to clojure and I do not understand how I convert it - It seems the api keys are referred in the header and I am using the clj-http library for the one line of code and attempting to get the syntax right . There is not much else except I receive 401 responses.

dev.4openID22:03:49

As a noobie in clojure I wish I knew how to look under the covers so to speak

seancorfield22:03:54

If you're really just getting started with Clojure, you might have better luck asking in #beginners -- folks there will show you more patience and will be more forgiving...

seancorfield22:03:43

This channel tends to assume a certain level of scientific method in debugging etc and some solid ability to read source code in libraries...