Fork me on GitHub
#beginners
<
2019-06-25
>
Nate Sutton02:06:35

with spec, how do you specify that a value could be one of a few keywords

Nate Sutton02:06:02

I guess just make it a set?

andy.fingerhut02:06:03

Use a Clojure set, e.g. #{:foo :bar :baz}

noisesmith17:06:20

nb this works with any plain values, except false or nil (I know the question was only about keywords)

Nate Sutton02:06:10

yeah, ok, cool

Paulo02:06:48

like this example from https://clojure.org/guides/spec: (s/def ::suit #{:club :diamond :heart :spade})

Nate Sutton02:06:57

I'm at this right now:

(s/def ::walls (s/map-of
                #{:north :south :east :west}
                #{:door :wall :exit :entrance}))

Nate Sutton02:06:17

the naming needs work. and I need to say those keys are all required

Nate Sutton02:06:23

but yeah, the sets work

Nate Sutton03:06:45

so with s/keys you have to have a spec named with the key you want it to use and then have that spec be the spec for the value. is there any way to detach those two? like I have one definition for an ::edge and I want the keys :north :south :east :west . I could add (s/def ::north ::edge) and repeat for all the directions but that seems silly

Nate Sutton03:06:00

I guess I could set the direction and count and distinct?

andy.fingerhut03:06:05

So you want a map that can have an arbitrary subset of the keys ::north ::south ::east ::west, and the type of the values associated with them will be the same for each?

Nate Sutton03:06:25

(s/def ::direction #{:north :south :east :west})
(s/def ::edge #{:door :wall :exit :entrance})
(s/def ::edges (s/map-of ::direction ::edge :distinct true :count 4))

Nate Sutton03:06:19

I guess the keys are already distinct, I'll nix that

andy.fingerhut03:06:06

You never want to allow a subset of the 4 possible keys, i.e. you want spec to check that all 4 are always present

andy.fingerhut03:06:02

I guess that is fine, if some of them are walls 🙂

Nate Sutton03:06:13

alternatively

Nate Sutton03:06:16

(s/def ::edge #{:door :wall :exit :entrance})
(s/def ::north ::edge)
(s/def ::south ::edge)
(s/def ::east ::edge)
(s/def ::west ::edge)
(s/def ::edges (s/keys :req-un [::north ::south ::east ::west]))

Nate Sutton03:06:23

but that kind of sucks

Nate Sutton03:06:42

like ::north should be ::north-edge

Nate Sutton03:06:52

this is my predicament

andy.fingerhut03:06:21

If you had 100 keys all with the same type, I would definitely agree. 4 seems on the edge of "eh, it gets the job done" to me, personally.

Nate Sutton03:06:14

the difficulty is that the vector in s/keys has two meaning, both the key name and the map value spec

Nate Sutton03:06:41

that seems bad to me

andy.fingerhut03:06:08

That seems pretty important when they have a large variety of specs for each of their corresponding values, doesn't it?

Nate Sutton03:06:10

it seems like a useful shorthand, but maybe not flexible enough

andy.fingerhut03:06:24

My understanding is that a spec key name is intended to be used throughout a possibly multi-process multi-machine system, and you want the name of that key to have the same kind of contents wherever it appears, in any map where it appears.

andy.fingerhut03:06:43

e.g. :com.mycompany/customer-id will always have the same kind of associated values, no matter where it is, no matter what it is combined with in whatever maps it ends up in.

Nate Sutton03:06:45

but when you have multiple types of cells with different values for the key :north

Nate Sutton03:06:02

this is a maze, which starts out using a grid of cells and then opening doors and entrances and exits in cells. but then in order to render the maze I need to flip it where the edges themselves become the coordinate plane. and suddenly the north/south/east/west become what type of wall or open space is in that direction for that segment of an edge

andy.fingerhut03:06:07

I do not know if I understand your use case well enough. Are you saying you might have 10 different kinds of cells, and for each of those 10 kinds, you want to constrain the set of allowed values differently for their :north key?

Nate Sutton03:06:23

well, no, probably 3

Nate Sutton03:06:57

my point is that when you're working with grids a bunch for different representations of things then north/south/east/west gets used for different values

Nate Sutton03:06:36

I guess I'm not modeling it in a way that spec likes or something

andy.fingerhut03:06:40

Sure. I do not have enough spec time under my belt to say whether it was designed to be bent in that direction, or whether you are stretching it unnaturally against its design.

Nate Sutton03:06:58

I'll just go with my hack that works I suppose

seancorfield03:06:03

@nate_clojurians If you go in a direction you get to a (type of) cell, yes?

seancorfield03:06:21

(assuming that direction is available to you)

Nate Sutton03:06:30

no, not for some of the grids

Nate Sutton03:06:25

it's difficult to explain without drawing it out

seancorfield03:06:46

When you are in a cell, each of the directions has a wall/door/passage/etc that determines whether you can go that way (and how you do it), yes?

andy.fingerhut03:06:49

If you had only 2 types of grids, with different sets of values of things you wanted to allow in the north direction for those 2 types, it seems worth asking whether having a :grid1/north and :grid2/north spec might be useful?

Nate Sutton03:06:06

@seancorfield for one of the grids, yes. for another, no

seancorfield03:06:16

And then there's the actual motion from cell to cell, so each direction takes you to a new type of cell...

Nate Sutton03:06:22

and probably about to have a third kind

seancorfield03:06:28

What are the different types of grids?

Nate Sutton03:06:38

I don't have names for them yet

Nate Sutton03:06:50

one is the cell grid you

Nate Sutton03:06:05

that you'd typically think of as a grid

Nate Sutton03:06:46

but another is an intermediate representation of that grid that breaks down that grid into subsections to determine where the walls are and what type of characters I'll need in each section

Nate Sutton03:06:13

and then that intermediate representation is transformed into a character grid (which is scaled with different parameters)

Nate Sutton03:06:05

three different kinds of grids, going from coarse cells to finer cells for display

Nate Sutton03:06:26

that intermediate representation can also be transformed into things other than text, which would need to be scaled differently

Nate Sutton03:06:59

but the point stands, different kinds of grids. you can call them cells in the other grids but they're not strictly cells in the sense of the initial maze generation

Nate Sutton03:06:23

so the types in specs get funny

Nate Sutton03:06:47

and I don't want to bind ::north to a type that will not be used in other grids

Nate Sutton03:06:23

I'm not even sure ::edge is going to work

Nate Sutton03:06:42

but that can be changed, I suppose, since it's not also the map key

didibus03:06:53

Hum... What are you using Spec for though ?

didibus03:06:19

All these intermediate steps don't necessarily need a spec. Or, not necessarilly a very granular spec

Nate Sutton03:06:29

I'm spec'ing out the data structure

didibus03:06:42

Yes, but why?

Nate Sutton03:06:49

because I find value in it

didibus03:06:59

As documentation?

Nate Sutton03:06:12

documentation and for checking my transformations each step of the way

didibus03:06:34

I'm saying that, because if your goal was validation, or generative testing, the choices for how and what to Spec might be different than other reasons to have the Spec

Nate Sutton03:06:05

yeah, I've barely touched on using it for generative testing so far

Nate Sutton03:06:14

I'm sure that'll color how I spec things

Nate Sutton03:06:08

I'm still in the exploration phase so this is helping me transform things from grid to grid and confirm my transformations are working

didibus03:06:34

So, I don't necessarily want to suggest that, because it gets hairy. And I'd wonder if it is really necessary, but you can obviously write spec generating code.

Brandon Olivier03:06:12

I'm trying to use Luminus to make a POST request, but whenever I try to access the body I get a #object[io.undertow.io.UndertowInputStream back. I've been trying to figure out what to do with that for like an hour and I still have no idea.

didibus03:06:28

Which could help you create all the combinations of s/keys you need

Nate Sutton03:06:24

the trouble I'm hitting is that s/keys doesn't allow you to split the key name from the key value definition

Nate Sutton03:06:44

so even if I auto-generated these things, I'd still have that problem

Nate Sutton03:06:34

this seems good enough for now:

(s/def ::direction #{:north :south :east :west})
(s/def ::edge #{:door :wall :exit :entrance})
(s/def ::edges (s/map-of ::direction ::edge :count 4))

didibus03:06:22

Hum... are you saying the same key will have different type of value in different context?

Nate Sutton03:06:28

I'll revisit it soon, I think. I'm on the IR step and realized I did bits of the original structure wrong and had to backtrack to make the IR transformation simpler

didibus03:06:06

I see, you could look into s/or and s/multi-spec

didibus03:06:03

They allow you to specify that a particular thing is one of many things, with multi-spec letting you define what based on a function.

Nate Sutton03:06:05

instead of ::north it could be :maze/north

Nate Sutton03:06:33

I suppose I could give it whatever namespace I wanted and then use :req-un

Nate Sutton03:06:01

I'm going to try that out

didibus03:06:36

Alright, I didn't fully understood exactly what you're trying to spec, but I'm glad my pointers helped

didibus03:06:21

Maybe if you posted an example of the data-structure in two different state?

Nate Sutton03:06:38

once I get the IR done I'll share

Nate Sutton03:06:38

that'll have to be for another night, though

Nate Sutton03:06:47

thanks again 🙂

didibus04:06:55

@brandon149 hum... this is a wild guess, I've never used Luminus, but try calling it with an @ in front

Brandon Olivier04:06:21

@didibus I'm still a total n00bie, what's the @ do?

Brandon Olivier04:06:33

I remember using it with atoms, is that going to be the same kind of thing?

didibus04:06:59

Well, normally it does a thing called deref. Which normally just mean get the value out of a thing.

didibus04:06:34

It might not be that what you have can be derefed, so it might not work

didibus04:06:46

But I thought there is a possibility that they would have designed it that way

Brandon Olivier04:06:51

That gives me an error java.lang.ClassCastException: class io.undertow.io.UndertowInputStream cannot be cast to class java.util.concurrent.Future

Brandon Olivier04:06:07

It's some kind of stream, but I'm not sure how to read it

didibus04:06:07

Ah ok, so ya no, it doesn't work like that then 😛

didibus04:06:11

It was a wild guess

Brandon Olivier04:06:20

most of the java code has something like while (stream.read() > -1)

Brandon Olivier04:06:28

but that doesn't really translate to clojure as far as I know

jason poage04:06:47

i want to start my program with a function other than -main, how do i do that without starting the repl first?

seancorfield04:06:06

lein run -m your.namespace/the-function -- see lein help run for more information

Nate Sutton04:06:16

or whatever the var is that holds the body if it's not named body

jason poage04:06:21

i am using lein

Brandon Olivier04:06:28

That's an IllegalArgumentException

Brandon Olivier04:06:31

No matching field found: read for class immutant.web.internal.ring.LazyMap

Brandon Olivier04:06:49

The real source of my trouble is that I can't figure out how to read an UndertowInputStream

didibus04:06:04

Well, depending what is on that stream, it could be complicated

Brandon Olivier04:06:13

I tried reading the docs for Undertow, but they weren't particularly helpful

didibus04:06:22

I feel like Luminous should have already parsed the response for you

Brandon Olivier04:06:25

which isn't surprising, given my lack of Java experience

Brandon Olivier04:06:31

@didibus you and me both, man

didibus04:06:07

haha, I'm just wondering if you are calling something lower level that returns the raw stream, and maybe there is another way to make a POST request that returns it parsed?

Brandon Olivier04:06:18

Given that I'm attempting to add a new ns of routes, it's probably that I forgot to add something required

Brandon Olivier04:06:33

I know the original one included some middleware, but I didn't include it

didibus04:06:52

Try calling it with slurp?

didibus04:06:14

That could read some type of streams

Brandon Olivier04:06:15

I tried that, it gives me an error something about how it's not a file

hiredman04:06:34

Clojure has arbitrary looping and the ability to call the read method

didibus04:06:59

Try calling on it first maybe?

didibus04:06:19

and then slurp ?

didibus04:06:08

Except you have an UndertowInputStream instead

Brandon Olivier04:06:07

So, after using , I end up having errors about not being able to JSON encoding a ByteArrayInputStream

Brandon Olivier04:06:12

which, again, seems pretty similar

didibus04:06:01

Something is weird, when you called (.read body) you got: immutant.web.internal.ring.LazyMap ?

didibus04:06:35

But when you call slurp on it you get an exception mentioning: UndertowInputStream ?

didibus04:06:56

You should be able to call .read on the UndertowInputStream. The issue is that it seems to be a byte stream. So you will just get a bunch of bytes, and then you need to know the encoding, so you can decode the bytes into whatever they represent

hiredman04:06:56

You 100% did not get lazymap back from calling read on a stream

hiredman04:06:40

Variations on a lazy maps are used in a number different ring adapters for the request or the headers map

hiredman04:06:19

aleph does this, I haven't used undertow in a long time, so it looks like they do this too

hiredman04:06:32

Lazy map being a map that doesn't have it's values filled in yet because the adapter is built on some kind of asyncy server that doesn't get the entire request at once

didibus04:06:05

I think you are probably missing a middleware which should decode the undertow stream or something. I really doubt Luminous doesn't handle the byte stream for you

Brandon Olivier04:06:13

I'm getting weirdly inconsistent results in CIDER and I'm not really sure why

Brandon Olivier04:06:34

now, I call .read on (:body req) and it comes back with 123

hiredman04:06:48

Inputsteam is and interface, and UndertowInputstream implements it

hiredman04:06:16

Correct, calling read on an inputstream returns and int

hiredman04:06:00

(because it can't return byte for reasons)

Brandon Olivier04:06:02

Well, that's good, I guess

hiredman04:06:17

Slurp will work on an inputstream

Brandon Olivier04:06:24

at least the results making sense now

hiredman04:06:41

My guess is you were confused earlier and called in on the request map or the header map

didibus04:06:09

Try slurp again

Brandon Olivier04:06:03

slurp is working now, but I don't understand why

hiredman04:06:21

Because you were confused earlier

Brandon Olivier04:06:37

Something weird is going on, because I get different results from the POST call, but they're not consistent

hiredman04:06:48

And were not doing what you thought you were doing

Brandon Olivier04:06:52

The code I have written right now is identical to something I wrote earlier

Brandon Olivier04:06:55

but the results aren't the same

didibus04:06:29

Could the post be async?

Brandon Olivier04:06:54

it's possible, I'm not particularly familiar with Luminus, and especially not Undertow

Brandon Olivier04:06:58

I think what was wrong with my debugging

Brandon Olivier04:06:17

was that, for reasons I don't understand, I need to evaluate the function I'm looking at

Brandon Olivier04:06:24

and then (restart) the app

Brandon Olivier04:06:35

I did not consistently do those things in that order

didibus04:06:39

Hum.. could be. But, it might be you just accidentally were calling slurp on the wrong thing.

Brandon Olivier04:06:05

It's possible. I'll ascribe the whole thing to my newbie status and hope to learn from it

1
magthe09:06:16

I'm playing around with the AWS API, so far I've been using amazonica, but I can't make head nor tails of how to set up a client, so I thought I'd try out Cognitect's AWS lib (https://github.com/cognitect-labs/aws-api). However, I'm completely failing to convince lein to download it, and there are only instructions for using deps.edn 😕 Any pointers?

andy.fingerhut09:06:39

Where it shows this in the README, to use for deps.edn:

andy.fingerhut09:06:42

{:deps {com.cognitect.aws/api       {:mvn/version "0.8.305"}
        com.cognitect.aws/endpoints {:mvn/version "1.1.11.568"}
        com.cognitect.aws/s3        {:mvn/version "722.2.468.0"}}}

magthe09:06:25

Yes, indeed... how do I translate that to something that lein will accept?

andy.fingerhut09:06:12

In a Leiningen project.clj file, that would become the following lines in that file:

andy.fingerhut09:06:18

:dependencies [
                 [com.cognitect.aws/api "0.8.305"]
                 [com.cognitect.aws/endpoints "1.1.11.568"]
                 [com.cognitect.aws/s3 "722.2.468.0"]
                 ]

dmaiocchi10:06:00

HI! have you ever done 2 projects in the same git repository with lein? :thinking_face: e.g I'm doing a client/server project, and I would like to have both on same repository instead of creating an organisation

tavistock12:06:36

you can probably do it in one file, i have in the past just used the cljs-build options to differentiate the 2. If you need more separation, eg diff dependencies etc (idk why you would) you can use profiles and the lein with-profile +whatever ... command

tavistock12:06:20

you would just run 2 different commands to start up each ones repl in develop, and make a command that builds both for deploy

dmaiocchi12:06:17

Thx I will try out

dmaiocchi13:06:47

I couldn't find out any documentation flag for it

dmaiocchi13:06:12

so I ended up to doing 2 directory each one with a seperate project.clj

dmaiocchi13:06:05

It is maybe not super beautiful but I didn't want to spend all the time researching it. The leinigein doc didn't have doc for that. And I think in JVM is possible to do it but I'm not the JVM expert 😁

dmaiocchi13:06:21

if anyone has a link feel free to share.. 🚀

tavistock14:06:25

can you link your project (if its opensource, etc)

tavistock14:06:26

make a new project doing lein new chestnut app-name and look at it, it is a template that serves up a routes and compiles a frontend https://github.com/plexus/chestnut

clj 1
dmaiocchi10:06:39

I'm looking currently if I can do it with only 1 project.clj file only or if i need to create directories with 2 project.clj

Pepe11:06:14

Hi. I'm trying to create a random matrix. I'm using the matrix.core library... (clojure.core.matrix/add (rand 5) (clojure.core.matrix/new-matrix 2 2)) But this gives me a matrix where every element is the same random number e.g. [[3.928259253657548 3.928259253657548] [3.928259253657548 3.928259253657548]]

Pepe11:06:47

I tried using repeatedly but the library doesn't handle it the same way normal clojure vectors handle it, so I haven't been able to make use of it

Pepe11:06:21

(`(clojure.core.matrix/emap + (repeatedly #(rand 2)) [ 1 1 1 ] )` this actually causes a crash, whereas normal map is fine)

tavistock12:06:34

(clojure.core.matrix/emap #(rand 2) (clojure.core.matrix/new-matrix 2 2)) may be what you are trying to do

Charles Fourdrignier12:06:48

Not sure that's a "good solution", but last time I needed to generate a matrix, I relied on clojure.spec + generators...

(def MATRIX-SIZE 100)
(s/def ::image-pixel #{0 1})
(s/def ::image-row (s/coll-of ::image-pixel :count MATRIX-SIZE))
(s/def ::image (s/coll-of ::image-row :count MATRIX-SIZE))

(gen/generate (s/gen ::image))

Pepe12:06:29

I've slightly modified @U0C7D2H1U code and it does indeed produce the expected matrix (clojure.core.matrix/emap #(+ % (rand 2)) (clojure.core.matrix/new-matrix 2 2))

Pepe12:06:17

I will stick to this for the time being since it accomplishes the task and it's moderately readable. Thanks guys

Sliverious16:06:23

Hi! I am looking to get some help on getting my installation of clojure working correctly. I am running Ubuntu 18.04 and have installed the prerequisite bash, curl, rlwrap and java, with the latter being

$ java -version
openjdk version "11.0.3" 2019-04-16
OpenJDK Runtime Environment (build 11.0.3+7-Ubuntu-1ubuntu218.04.1)
OpenJDK 64-Bit Server VM (build 11.0.3+7-Ubuntu-1ubuntu218.04.1, mixed mode, sharing)
Some basic functionality works fine, like (def x 7) or (* 8 8), but while working through the official guide (https://clojure.org/guides/learn/syntax) I often walk into java exceptions for code that does work on https://repl.it/languages/clojure, like
Clojure 1.10.0
user=> (def anex (try (/ 1 0) (catch Exception ex ex)))
#'user/anex
user=> (clojure.stacktrace/root-cause anex)
Execution error (ClassNotFoundException) at java.net.URLClassLoader/findClass (URLClassLoader.java:471).
clojure.stacktrace
Does anybody know what exception is actually occurring and how to fix this? I could only find an SO thread mentioning java 8+, but I think I have that installed (I am unsure, I'm not a java programmer). Thanks in advance.

hiredman16:06:46

you need to require clojure.stacktrace

Sliverious16:06:47

How do I require that library?

hiredman16:06:24

it is a namespace, and it is part of clojure, so you just need to (require 'clojure.stacktrace)

hiredman16:06:16

some of the namespaces that ship with clojure are required elsewhere so end up being sort of implicitly loaded, but the set of namespaces that get loaded like that have changed overtime so it is a best practice to require them if you need them

Sliverious16:06:58

Right, that clarifies a lot.

Sliverious16:06:34

The example I gave also works when I require the library, so thanks for helping!

Sliverious16:06:17

I guess http://repl.it just autoloads a bunch of namespaces then. 😛

hiredman16:06:05

it likely loads clojure.stacktrace for pretty printing exceptions or something

hiredman16:06:25

if code is loaded, you can use it without requiring it

Sliverious16:06:19

That seems reasonable.

NoahTheDuke17:06:32

in javascript, i can say cost game = new Game(); let card = new Card(game); game.cards.push(card);, which creates a recursive relationship between the two objects. It's useful so inside the Card class, you can say if (this.game.method()) { do_stuff(); }. Is that possible in Clojure? If not, what are the best idioms to handle that kind of relationship?

noisesmith17:06:36

usually instead of recursively nested values, I'd use a flattened non-recursive structure that describes both relationships

noisesmith17:06:04

there's no construct or shortcut to create mutual nesting, so you need to use some trick with laziness or mutation

noisesmith17:06:16

and it's not worth the headache or complexity IMHO

noisesmith17:06:43

(by flattened non-recursive structure describing relationships I of course mean adjacency list)

lilactown17:06:15

to reiterate what noisesmith said: you can do this using mutable values but it is very un-idiomatic

noisesmith17:06:34

so foo.bar.parent_foo = foo is instead {:foo {:bar :bar} :bar {:parent_foo :foo}}

NoahTheDuke17:06:48

hm! that's an interesting idea!

noisesmith17:06:13

the drawback is that instead of indexing the adj-list directly, you need a function that knows how to walk it

noisesmith17:06:45

the advantage is you can describe complex recursive relations in immutable data, and the editing operations are much less error prone than the mutable version

noisesmith17:06:04

also you don't run into stack overflow errors printing it naiively :D

NoahTheDuke17:06:15

heh That is a benefit!

john18:06:12

Another option: let Game and Card instances have unique IDs so they can store references to external instances

noisesmith18:06:44

in practice it's more complex than my example above - more like {:edges {:foo #{:bar}} :nodes {:bar {:parent-foo :foo}}

NoahTheDuke18:06:04

that's another interesting idea, @john. i'll have to think about that!

tavistock18:06:25

you can look at https://github.com/aysylu/loom for inspiration

💯 1
tavistock18:06:25

although it is pretty heavy and might not be too much

john18:06:55

Taking that route, you'd have to manage those references in some state somewhere

john18:06:11

But that can be done functionally

noisesmith18:06:10

@john yeah - the adj-list stores the references and the data in one immutable structure for example, but can use unique ids instead of symbolic names

noisesmith18:06:21

then it's effectively the same thing

NoahTheDuke18:06:46

as background: i've been playing with porting a pipeline/queue system from a game engine written in javascript to clojure, for potential use in a card game i maintain. the idea is that the pipeline queues "steps" (which themselves can contain pipelines), and they share common interfaces such as being able to say pipeline.continue() which will then grab the first step in the queue and call that step.continue() which then looks to see if it has it's own pipeline or whatever

NoahTheDuke18:06:19

i feel like i'm going about this the wrong way, but i still feel very new to functional/immutable programming, so trying to make stuff like this work has been kind of breaking my brain lol

noisesmith18:06:01

@nbtheduke another thing to look at for this kind of game state manipulation are Entity Component Systems, which translate very nicely into immutable data in hashmaps

NoahTheDuke18:06:42

That's pretty close to what the javascript engine does, and is definitely not what the clojure engine does, lol

noisesmith18:06:55

and for simple mechanics of queues, use clojure.lang.PersistentQueue/EMPTY which sadly doesn't have a nice looking data literal

noisesmith18:06:14

of course nothing prevents you from doing (def | clojure.lang.PersistentQueue/EMPTY)

NoahTheDuke18:06:28

lol |, that's pretty clever

john18:06:08

@noisesmith do you think he could model his game on core.async machinery? Sounds like he wants to put channels in channels, or some similar abstraction

noisesmith18:06:54

channels are an io mechanic, and queues are a data structure

noisesmith18:06:14

they often go together but not always - I'd hold off on async until the domain requires it honestly

noisesmith18:06:32

(cmd)user=> (def | clojure.lang.PersistentQueue/EMPTY)
#'user/|
(cmd)user=> (-> | (conj :a :b :c) (pop) (conj :d) seq)
(:b :c :d)

john18:06:42

Aye, harder for me to think functionally in core.async too

noisesmith18:06:01

right - it's an IO framework not a functional abstraction

NoahTheDuke18:06:53

I looked into the PersistentQueue, but my issue with it is there's no easy method for prepending things

noisesmith18:06:08

you need a double ended queue?

noisesmith18:06:26

I mean, usually queue means fifo, which PersistentQueue does

NoahTheDuke18:06:44

potentially! i'm not quite sure

NoahTheDuke18:06:46

Like, I perform an action that then requires reactions, so i'd like to put the reactions before the rest of the queued actions

noisesmith18:06:17

there's DeQueues built into the vm, but none immutable, and none in clojure itself

noisesmith18:06:41

of course via into and concat you can force it

Nate Sutton18:06:47

priority queue of some sort?

noisesmith18:06:48

(ins)user=> (def q (-> | (conj :a :b :c) (pop) (conj :d)))
#'user/q
(ins)user=> (seq q)
(:b :c :d)
(ins)user=> (into | cat [[:a] q])
#object[clojure.lang.PersistentQueue 0x736d6a5c "[email protected]"]
(ins)user=> (seq *1)
(:a :b :c :d)

noisesmith18:06:10

the prepends are more expensive, but they are possible, and if they are rare it might be worth it(?)

NoahTheDuke18:06:54

they would not be rare

noisesmith18:06:33

you could look into finger-trees if you are stuck on immutability https://github.com/clojure/data.finger-tree or use a built in Deque https://docs.oracle.com/javase/7/docs/api/java/util/Deque.html

Nate Sutton18:06:29

you could use async channels for them, one for the normal queue and one for reactions, then use alts! with :priority so that it always tries to read from the reaction queue first

Nate Sutton18:06:19

oh, that's good

Nate Sutton18:06:29

not sure how you'd manage the peek + pop in an atomic operation, and add a wait like a queue might provide

Nate Sutton18:06:05

also enqueueing new stuff safely with concurrency

noisesmith18:06:38

there's no issue with data safety in concurrency in js

Nate Sutton18:06:35

sure, but if two contexts do a peek at the same time...

Nate Sutton18:06:41

in the priority map

noisesmith18:06:46

js has no "at the same time"

Nate Sutton18:06:09

I don't understand when exactly is switches contexts in js

noisesmith18:06:19

it's via returns / callbacks

noisesmith18:06:29

so it's cooperative, there's no interruption

Nate Sutton18:06:49

so the reactor is really, truly blocked if the callback does something like loop infinitely

noisesmith18:06:04

there are concurrency frameworks, but they are driven by providing and returning from callbacks, they don't interrupt code

Nate Sutton18:06:01

well, that does simplify it

noisesmith18:06:03

in jvm clojure where we do have arbitrary interruptive context switches, the combination of immutable data structures and safe containers like atom / ref / agent are sufficient if used correctly

noisesmith18:06:30

but there's a whole class of errors (all the data integrity stuff) that just doesn't happen in a single thread like js

Nate Sutton18:06:32

I've done a ton of concurrency work just never in js ¯\(ツ)

Nate Sutton18:06:55

the cooperative switching is wild

Nate Sutton18:06:04

I guess that's why they have the await syntax now

Nate Sutton18:06:20

so they can yield to the reactor to wait for a callback to finish firing

Nate Sutton18:06:36

or whatever await is called on, really

noisesmith18:06:40

so that data.priority-map linked above - another thread can't corrupt it because it's immutable, and the workings of atom would ensure that the replacement of one reference to an immutable map by another is done safely

noisesmith18:06:03

@nate_clojurians precisely, it's classic cooperative concurrency

Nate Sutton18:06:28

I know they can't corrupt it, my concern was losing items that are pushed onto it or double-popping items off of it

noisesmith18:06:43

right, and using an atom correctly prevents this

Nate Sutton18:06:16

I don't know how you'd use an atom for that, but I haven't played with them much

noisesmith18:06:18

and in js, it's simpler because it suffices to not return until the data is "coherent" by your definition

hiredman18:06:56

Use clojure.core/compare-and-set!

noisesmith18:06:04

@nate_clojurians the tl;dr version is your function passed to swap! is written so that it does all the operations for an update in one go, and either returns a valid state or errors

noisesmith18:06:20

even swap! suffices, but swap-vals! is useful sometimes as well

hiredman18:06:25

Use compare-and-set!

hiredman18:06:36

Swap! Is not what you want

hiredman18:06:55

You don't need to shoe horn it in to swap!

NoahTheDuke18:06:15

What's the difference between swap! and swap-vals!?

Nate Sutton18:06:41

yeah, compare-and-set looks the simplest. deref and peek/pop and try to write the new queue/map back to the atom

NoahTheDuke18:06:00

is it just that swap-vals! returns both old and new?

hiredman18:06:13

I've also got a mutable queue that lets you use cas to pop values

dpsutton18:06:06

(I have a PR to that queue )

dpsutton18:06:19

I didn't mean to offend. I enjoyed studying the code and linked paper. I thought i identified a small bug and sent a patch to fix it. Didn't mean to annoy

hiredman18:06:42

oh no, I just haven't looked at it in a while

dpsutton18:06:04

ah ok. yeah i mentioned it now just because it was on your mind 🙂

credulous18:06:29

Hi. I’m doing something boneheaded with jdbc.next. Everything works fine when going against a local database, but when I try to connect to a remote I get exceptions saying my connection string is probably wrong. I’m creating a datasource by doing something like this:

(->> {         :user "myuser"
         :password "very-secure-pw"
         :dbname "energy"
         :dbtype "mysql"
         :host ""
         }
         jdbc/get-datasource
         (reset! datasource))

credulous18:06:46

I can connect to mysql using the command line just fine at that host: mysql -u 'myuser'@'machinename' -p -h energy works fine

credulous18:06:30

I’ve tried changing the user in the config map, using “myuser”, “[email protected]” and “‘myuser’@‘machinename’”

credulous19:06:45

next.jdbc/get-datasource is returning nil with those parameters.

seancorfield19:06:03

@credulous I'm not sure how get-datasource can return nil at all here. It should either throw an exception or return a javax.sql.DataSource object. It would also really help us to help you if you could share the exact exception you are getting for "my connection string is probably wrong"... Lots of exceptions are possible if you don't have things set up correctly 🙂

credulous20:06:41

Thanks Sean. This is the exception I’m seeing: (the first line is just from a (println @datasource) in my code)

credulous20:06:14

19-06-25 20:44:02 MacBook-Pro.hitronhub.home INFO [alexandria.services.mysql:95] - Executing sql against datasource [email protected]
19-06-25 20:44:03 MacBook-Pro.hitronhub.home ERROR [alexandria.services.mysql:98] - SQLException The connection string may not be right. Please visit portal for references.
        com.mysql.cj.jdbc.exceptions.SQLError.createSQLException (SQLError.java:129)
        com.mysql.cj.jdbc.exceptions.SQLError.createSQLException (SQLError.java:97)
        com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException (SQLExceptionsMapping.java:122)
        com.mysql.cj.jdbc.ConnectionImpl.createNewIO (ConnectionImpl.java:832)
        com.mysql.cj.jdbc.ConnectionImpl.<init> (ConnectionImpl.java:456)
        com.mysql.cj.jdbc.ConnectionImpl.getInstance (ConnectionImpl.java:240)
        com.mysql.cj.jdbc.NonRegisteringDriver.connect (NonRegisteringDriver.java:207)
        java.sql.DriverManager.getConnection (DriverManager.java:664)
        java.sql.DriverManager.getConnection (DriverManager.java:208)
        next.jdbc.connection/get-driver-connection (connection.clj:78)
        next.jdbc.connection/get-driver-connection (connection.clj:74)
        next.jdbc.connection/url+etc/reify--1548 (connection.clj:145)
        next.jdbc.connection/make-connection (connection.clj:162)
        next.jdbc.connection/make-connection (connection.clj:152)
        next.jdbc.connection/eval1564/fn--1565 (connection.clj:181)
        next.jdbc.protocols/eval1358/fn--1359/G--1349--1366 (protocols.clj:24)
        next.jdbc.result-set/eval2004/fn--2012 (result_set.clj:449)
        next.jdbc.protocols/eval1390/fn--1421/G--1381--1430 (protocols.clj:33)
        next.jdbc/execute! (jdbc.clj:165)
        next.jdbc/execute! (jdbc.clj:155)
        alexandria.services.mysql/execute! (mysql.clj:96)
        alexandria.services.mysql/execute! (mysql.clj:91)
        alexandria.services.mysql/query (mysql.clj:102)
        alexandria.services.mysql/query (mysql.clj:101)
        alexandria.services.mysql$eval11520.invokeStatic (:105)
        alexandria.services.mysql$eval11520.invoke (:105)
        clojure.lang.Compiler.eval (Compiler.java:7177)
        clojure.lang.Compiler.load (Compiler.java:7636)
        user$eval11516.invokeStatic (:1)
        user$eval11516.invoke (:1)
        clojure.lang.Compiler.eval (Compiler.java:7177)
        clojure.lang.Compiler.eval (Compiler.java:7132)
        clojure.core/eval (core.clj:3214)
        clojure.core/eval (core.clj:3210)
        clojure.main/repl/read-eval-print--9086/fn--9089 (main.clj:437)
        clojure.main/repl/read-eval-print--9086 (main.clj:437)

credulous20:06:47

The comment about datasource being nil was a red herring, sorry

seancorfield20:06:32

And the remote MySQL instance is running... where?

seancorfield21:06:23

Looks like Azure, based on that error.

seancorfield21:06:30

Ah, OK, this is due to Azure's slightly weird username rules. Which version of next.jdbc are you using? (1.0.0 "gold" should print something more informative for the reified datasource but it will depend on how, exactly, you are printing it -- I would have expected the datasource to print the actual URL being used.

seancorfield21:06:02

For example, with next.jdbc 1.0.0 (as opposed to the RC):

(! 573)-> clj -Sdeps '{:deps {seancorfield/next.jdbc {:mvn/version "RELEASE"}}}'
Clojure 1.10.1
user=> (require '[next.jdbc :as jdbc])
nil
user=> (def ds (jdbc/get-datasource {:dbname "mydb" :dbtype "mysql" :user "admin" :password "secret" :host ""}))
#'user/ds
user=> (println "this is the datasource" ds)
this is the datasource #object[next.jdbc.connection$url_PLUS_etc$reify__397 0x2a415aa9 jdbc:]
So I would expect to see the URL in the output from your println call.

seancorfield21:06:19

Based on the Azure docs, I would expect :user to need to be [email protected] (which I know you said you tried but perhaps you could try it again?)

credulous15:06:26

Thanks, Sean, I really appreciate the help. The issue was indeed the username. Your deduction skills are amazing.

credulous15:06:29

And sorry for not replying right away, my tranquil home suddenly turned into a toddler-bedtime warzone

noisesmith19:06:26

you might get a better answer on #sql, but my first instinct is that a db spec and a datasource are two ways to get a connection, and you are asking for the datasource out of a db spec

noisesmith19:06:16

so I'd expect get-connection which would work on a map like yours, or a datasource

ghadi19:06:56

or some ssl auto-negotiation that the cmdline does that the jdbc driver does not

noisesmith19:06:41

maybe it is about ssl etc.

credulous19:06:53

Thanks, maybe I’ll mess with some ssl config.

ghadi19:06:28

I'd start with the MySQL JDBC driver docs

ghadi19:06:41

you may just have to put mysqls:/ or something simple

ghadi19:06:53

assuming this is a DB public on the internet

seancorfield19:06:07

clojure.java.jdbc and next.jdbc make no assumptions about SSL or not -- they leave that entirely up to the JDBC driver.

seancorfield19:06:52

@credulous I'm happy to dig into more detail about this in #sql but if you can tell us exactly what error you're getting when connecting to remote, that's going to be a good starting point.

seancorfield19:06:53

In next.jdbc, you can (and should) create a datasource from a hash map (unless you're building your own pooled datasource via Hikari/c3p0 etc), so the code is correct in that regard.

👍 1
seancorfield19:06:23

(then you -- or next.jdbc -- can create connections from the datasource, via get-connection)

noisesmith19:06:22

ahh- that was my confusion, I thought of a datasource vs. connection config as alternate inputs to get-connection, I guess this taxonomy makes more sense

seancorfield19:06:15

It was kind of muddy in clojure.java.jdbc since everything was a hash map. next.jdbc works on the Java types (`java.sql.Connection`, javax.sql.DataSource)

Joachim Smith19:06:39

can anyone explain why

(into #{})
is faster than
(set)
? If both are the same, why ever use
(set)
?

mss19:06:50

hey all, wondering what the idiomatic way to traverse a tree and build up a seq of the results of meta calls on each node. kind of like reduce combined with postwalk. I’m sure there’s a simple solution, but I can’t quite wrap my head around an idiomatic way to do that

seancorfield19:06:39

@mayanaze I'd have to see some specific code to comment on that. I just ran a few quick tests and didn't much difference between (set coll) and (into #{} coll)...

seancorfield19:06:50

Based on the source, both will use reduce with conj! over a transient collection...

Joachim Smith20:06:47

@seancorfield thanks for the reply. the test was i was running is pretty simple. run both a couple of times and into is consistently faster

Joachim Smith20:06:26

the def is so the repl doesn't need to print 10000000 times

seancorfield20:06:35

Did you use that actual code? nums will be realized the first time it is used, but then it will be a fully-realized sequence in the second call.

andy.fingerhut20:06:01

It might not be relevant, but what OS, JVM version, and Clojure version are you running?

andy.fingerhut20:06:12

and what kinds of run time results are you seeing?

andy.fingerhut20:06:25

When I run both of those defs several times with macOS 10.13.6, Oracle JDK 1.8.0_192, Clojure 1.10.1, the first one was about 21 seconds, but after that both of them varied between 7 to 10 seconds on different runs, where the longer times are likely due to more GC time when the JVM needed to free up some memory. The first time being much longer is likely due to the JVM byte code not being JIT'ed yet.

andy.fingerhut20:06:39

oh, yes, and the reason that Sean mentioned -- range and take do not actually create the sequence until you force them to, which will happen the first time you run one of the last two def's.

Joachim Smith20:06:01

i understand, which is why i run both tests multiple times and ignore the first one

andy.fingerhut20:06:08

And you see one of those two being consistently faster than the other on your system, across multiple runs? More than the variation in times of the same expression across multiple runs?

Joachim Smith20:06:13

here's a more robust test

hiredman20:06:23

things have changed

andy.fingerhut20:06:31

So regarding your original question, I would think that some people might prefer (set ...) over (into {} ...) for code readability reasons, although the difference is minor. I would say that while there are differences in the run times there, and there might even still be with Clojure 1.10, they are within about 10% of each other, so nothing I would lose sleep over. If there is a straightforward root cause that one of them really is consistently faster than the other, a performance-enhancing patch for the other one might be interesting.

Joachim Smith20:06:00

@hiredman have the functions changed?

hiredman20:06:16

the internals, yes

hiredman20:06:48

even the internal implementation of range is very different

hiredman20:06:03

(not sure exactly, that may be 1.6 but I think it might have been 1.7)

hiredman20:06:19

I think the big shake up of fast path reduce implementations, fast iterators, and faster range implementation are all post 1.6

ghadi20:06:15

1.7 is when we did that

ghadi20:06:34

but this is a non-sensical benchmark in the first place

ghadi20:06:44

no real world code will be dominated by these costs

hiredman20:06:12

into processes the collection via reduce, which is fast pathed now

hiredman20:06:24

I think set may still walk the seq, not sure

ghadi20:06:32

so what if there is a minor performance difference, either way

Joachim Smith20:06:56

another example:

Joachim Smith20:06:57

(time (dotimes [_ 100] (vec (range 100000))))

Joachim Smith20:06:05

(time (dotimes [_ 100] (into [] (range 100000))))

ghadi20:06:14

are you still on 1.6?

andy.fingerhut20:06:56

Unless you are looking for history of performance changes, the latest Clojure version is probably of most interest here.

ghadi20:06:23

as @hiredman mentioned, range + reduce/into has a fast path

Joachim Smith20:06:40

so into is generally faster?

ghadi20:06:01

no, because vec now uses IReduceInit under the covers

ghadi20:06:25

But I'd stop obsessing over faster, make the code clear first

Joachim Smith20:06:44

unfortunately faster is better

Joachim Smith20:06:25

thanks everyone who helped out

andy.fingerhut20:06:02

I mean, if some code is in the hot spot of your system or program, and 10% differences there are important to you, measure a few variations and go for it. There are definitely some constructs in Clojure that make a much bigger difference than 10% on performance, and those are worth profiling for and improving -- where it actually matters to you.

hiredman20:06:16

neither of those thing is great for performance

hiredman20:06:27

building a new vector object from something else

hiredman20:06:46

like, if you don't do that at all, imagine how much faster things will be

PaulGureghian22:06:46

Hi. Clojurians

👋 4
1
PaulGureghian23:06:46

Hi. Trying to install Clojure on Win7. I changed the script to use "Tls" but now I get this error and I have tried some fixes but they didn't work.

noisesmith23:06:56

@paulgureghian is your goal specifically to use the clojure / clj command line tools, or to use the clojure language?

noisesmith23:06:42

because iirc with windows leiningen is much more mature, and it can load and run the clojure repl / compiler (as long as you have a jvm available)

seancorfield23:06:44

@paulgureghian If you're determined to get the PowerShell alpha installer working on Windows 7 (which I think would be great), the #clj-on-windows channel will probably be a better place to ask for help since that's where the folks who've worked on that PS script hang out.

PaulGureghian23:06:45

Can I do both ?

seancorfield23:06:20

If you just want to try out Clojure on Windows 7, I agree with @noisesmith that Leiningen is probably going to be the easier path right now.

noisesmith23:06:44

you can use both - but lein is more likely to just work out of the box, and using lein won't be a blocker to following tutorials you'll find online

noisesmith23:06:09

whereas deps.edn (the basis for clj / clojure commands) is newer, and less documentation assumes you would be using it

PaulGureghian23:06:12

Whats the url ?

noisesmith23:06:59

you can download and run lein.bat

PaulGureghian23:06:02

Is it like a modern distribution of Clojure ?

noisesmith23:06:18

it's a package manager that assumes you are going to run clojure

noisesmith23:06:54

you can use the latest clojure version with lein, and it knows how to pull in other libraries, run tests, create a standalone jar for distribution, etc.

noisesmith23:06:27

clojure itself is a java library, and lein uses a config file to arrange some set of dependencies from a cache, and start the process

seancorfield23:06:33

Leiningen has been around for a long time so it's the most battle-tested option on Windows, right now.

PaulGureghian23:06:19

I can use Clojure in the terminal and the language ?

noisesmith23:06:27

right, that's what lein is for

noisesmith23:06:52

(with a focus on configuring individual projects)

noisesmith23:06:18

eg. once you install lein.bat you can use lein repl to get a clojure interactive repl

noisesmith23:06:23

(in a terminal)

PaulGureghian23:06:31

Using the language means using Atom or VSCode with the Clojure extension ?

noisesmith23:06:03

there are extensions to atom and vscode to connect to nrepl, which is the network repl process lein starts

PaulGureghian23:06:40

Yup. Thats what I have

noisesmith23:06:06

yeah, that actually assumes you are using lein and wouldn't work with clj out of the box

PaulGureghian23:06:46

I assume Clojure on Mac / Linux is better ?

noisesmith23:06:15

somewhat - but since clojure is all java, and java is portable, it all just works once you get a jvm and lein running

noisesmith23:06:38

(with some gotchas around eg. emacs which prefers to run under linux / mac, but it doesn't sound like you'd have that problem)

PaulGureghian23:06:37

Ok. Lets try Lenin

leiningen 1
noisesmith23:06:18

there's some cool things going on with clj / deps.edn but especially with windows, it's probably not the right starting place for a beginner

PaulGureghian23:06:51

What is the right starting place ? lein ?

noisesmith23:06:00

yes, use lein to start

PaulGureghian23:06:20

I'll let ya know how it goes.

PaulGureghian23:06:39

Will Oracle Java be ok ?

noisesmith23:06:38

yes, as long as you use 1.7 1.8 or newer

andy.fingerhut23:06:12

The very latest Clojure (1.10) now requires Java 1.8 or higher.

noisesmith23:06:48

but really you should probably use 1.11 if you can, right?

andy.fingerhut23:06:11

1.8 is still pretty solid, but for production use I expect 1.8 will start getting harder and harder to find support for. For personal use, no problem.