Fork me on GitHub
#beginners
<
2018-05-16
>
stardiviner00:05:16

Hmm, I have to try this with that commit? Let me take a try again

seancorfield00:05:12

@stardiviner The SHA is in that article...

seancorfield00:05:37

...I think this will become quite a common way to publish little utilities. The Cognitect test-runner works like that, as do depstar and juxt.pack I think? Also the clj-new utility I published.

stardiviner00:05:30

Hmm, still same error with:

clj -Sdeps "{:deps {org.clojure/tools.deps.alpha {:git/url \"" :sha \"d0508cfacfbb1285ac4c3706688176baceb3f83b\"}}}"

stardiviner00:05:00

tools.deps.alpha.repl namespace still not available, neither add-lib.

stardiviner00:05:39

@seancorfield That's great!! Clojure will be easier to dynamically load lib will be soon.

stardiviner00:05:30

BTW, why use tools.deps.alpha instead of tools.deps, because it's in Alpha development status?

seancorfield00:05:56

@stardiviner Where did you get that SHA from? The article says d492e97259c013ba401c5238842cd3445839d020

seancorfield00:05:43

Yes, that article.

stardiviner00:05:03

Should I use this instead?

seancorfield00:05:07

The correct SHA is in the article.

seancorfield00:05:07

Per the article

(! 944)-> clj -Sdeps "{:deps {org.clojure/tools.deps.alpha {:git/url \"" :sha \"d492e97259c013ba401c5238842cd3445839d020\"}}}"
Checking out:  at d492e97259c013ba401c5238842cd3445839d020
Clojure 1.9.0
user=> (use 'clojure.tools.deps.alpha.repl)
nil
user=> (add-lib 'org.clojure/core.memoize {:mvn/version "0.7.1"})
true
user=> 

seancorfield00:05:01

As Alex said, the exact command-line to use is in that article 🙂

stardiviner00:05:46

Ahu, that's weird, where did I got this commit SHA? I must be copied from somewhere into my Org

seancorfield00:05:10

The SHA you used looks like the most recent commit on the master branch.

seancorfield00:05:27

@alexmiller One problem with that article as it stands is the formatting of the code means you can't just copy'n'paste it into a terminal... you seem to get spurious characters:

(! 943)-> clj -Sdeps "{:deps \
>                {org.clojure/tools.deps.alpha \ 
>                 {:git/url \"" \
>                  :sha \"d492e97259c013ba401c5238842cd3445839d020\"}}}"
Error while parsing option "--config-data {:deps                {org.clojure/tools.deps.alpha \\ \n                {:git/url \""                  :sha \"d492e97259c013ba401c5238842cd3445839d020\"}}}": java.lang.RuntimeException: Map literal must contain an even number of forms

stardiviner00:05:09

Yeah, I got this problem before, after tried some times, Then I remove "`\`", turn this command into one-line.

stardiviner00:05:26

What If I want to specify a branch, instead of :sha what keyword should I use? I tried :branch "add-lib" seems does not work.

seancorfield00:05:01

I think those \ aren't necessary -- contrast with https://github.com/seancorfield/clj-new/blob/master/README.md which only has \ outside the quotes (and does copy'n'paste properly).

seancorfield00:05:49

You can only use :sha. But you can always find the latest SHA in any given branch by looking on GitHub.

stardiviner00:05:21

right, use '...' can avoid those \

seancorfield00:05:21

@stardiviner The nice thing is that you can also write multi-file gists on GitHub and pull code from those in via clj -Sdeps, for example https://gist.github.com/seancorfield/6e8dd10799e9cc7527da5510c739e52f

stardiviner00:05:30

Yes, you remind me that, I did remind that you shared me many useful links. At that moment, I can't understand some of them. Now I can see more. Thank you.

stardiviner00:05:06

Aha, I found your shared this link, I recorded it into my Org.

seancorfield00:05:07

That's my ~/.clojure/deps.edn file full of useful aliases for clj

seancorfield00:05:29

I should probably annotate that file with examples of using the aliases and links to the original repos...

stardiviner00:05:55

That's better.

seancorfield00:05:36

No, I use Atom and ProtoREPL.

seancorfield00:05:56

I used to use Emacs and CIDER for years, but switched about 18 months ago (I think).

stardiviner00:05:12

No other means, I just want to inject tools.deps.alpha dependency into CIDER jack-in.

stardiviner00:05:19

So I asked you by the way.

seancorfield00:05:12

Looks like CIDER only supports Maven coordinates.

stardiviner00:05:29

Yes, I take a peek in this option source code, and other used places of functions, only support Maven version numbers. I will go to propose an feature request on CIDER.

stardiviner01:05:46

After checked out clj-new, It's an smart utility. 🙂

stardiviner01:05:41

After I do (add-lib 'incanter/incanter-core {:mvn/version "1.9.2"}) I try to require incanter with (require 'incanter.core) but got error FileNotFoundException Could not locate incanter/core__init.class or incanter/core.clj on classpath. clojure.lang.RT.load (RT.java:463) . I also tried (require 'incanter/incanter.core) , (require 'incanter-core/incanter.core). Also tried google "add-lib incanter", no useful results. Can someone explain the principle behind this? In an example, use (add-lib 'org.clojure/core.memoize {:mvn/version "0.7.1"}) and (require 'clojure.core.memoize) is fine. what's the difference?

stardiviner01:05:52

I also tried (require 'incanter.incanter.core), not work too.

stardiviner01:05:55

I start REPL with: clj -Sdeps '{:deps {org.clojure/tools.deps.alpha {:git/url \"https://github.com/clojure/tools.deps.alpha.git\" :sha \"d492e97259c013ba401c5238842cd3445839d020\"} org.clojure/tools.nrepl {:mvn/version \"0.2.13\"} refactor-nrepl {:mvn/version \"2.4.0-SNAPSHOT\"} cider/cider-nrepl {:mvn/version \"0.18.0-SNAPSHOT\"}}}' -e '(require (quote cider-nrepl.main)) (cider-nrepl.main/init [\"cider.nrepl/cider-middleware\"]) (use 'clojure.tools.deps.alpha.repl)'

Alex Miller (Clojure team)01:05:41

I have had issues trying to use the add-lib stuff over an nrepl connection

seancorfield01:05:22

It works from a plain console REPL:

(! 970)-> clj -Sdeps "{:deps {org.clojure/tools.deps.alpha {:git/url \"" :sha \"d492e97259c013ba401c5238842cd3445839d020\"}}}"
Clojure 1.9.0
user=> (use 'clojure.tools.deps.alpha.repl)
nil
user=> (add-lib 'incanter/incanter-core {:mvn/version "1.9.2"})
true
user=> (use 'incanter.core)
nil
user=> 

Alex Miller (Clojure team)01:05:31

add-lib works by finding the highest DynamicClassLoader and injecting the urls there but it seemed like I was seeing different URLClassLoaders each time when over nrepl

stardiviner01:05:10

Hmm, I see. @alexmiller I know you're the maintainer of tools.deps.alpha, do you think it is fixable?

Alex Miller (Clojure team)01:05:28

don’t know. I think CIDER already has something to do this?

seancorfield02:05:47

@alexmiller I'm curious, looking at things like clj and test-runner and transcriptor -- and how I've seen Cognitect folks show off code during talks, I get the impression that minimal tooling is favored over "heavy" tooling?

seancorfield02:05:16

Do any of the Cognitect folks use CIDER? Or do most of them stick to inf-clojure or whatever it's called?

stardiviner02:05:48

@alexmiller Yes, clj-refactor can hotload dependency, but cljr does not work if without project.

stardiviner02:05:46

I sometimes launch CIDER REPL without project for Emacs Org-mode ob-clojure for simple literate programming support.

Alex Miller (Clojure team)02:05:49

I personally use Cursive with an nrepl in context of Leiningen or Maven projects pretty often. I also use Emacs with CIDER and nrepl. And I use clj on the command line pretty frequently.

seancorfield02:05:51

I've been playing with unravel/unrepl and really like that simplicity -- I can start up an app on QA (or even production - shhh!) with a Socket REPL, tunnel to the server from my laptop, then run unravel against that and preload compliment.core for completion and then I have a pretty nice user experience.

stardiviner02:05:54

interesting, I will take a play of unravel/unrepl.

seancorfield02:05:04

(caveat: compliment support is somewhat experimental at the moment with unravel I think!)

dpsutton02:05:49

That sounds super nice. I like the idea of a little sugar around the repl to load files and maybe some jump to definition. But I don't use much of the features

xtreak2907:05:29

Has anyone used Clojure spec with functions that involve database or some external service? I don't know if it can be used only for pure functions (functions that don't to IO) and would like to do something like fuzzing the function. I don't if it's right to use Clojure spec for this.

seancorfield07:05:11

@xtreak29 Could you elaborate?

seancorfield07:05:48

Specs are, essentially, static -- non-parameterized -- so I'm not sure what you're asking about?

xtreak2907:05:52

In the clojure spec guide there was a section where generative testing was done with random number function (https://clojure.org/guides/spec#_testing). I was wondering if the same can be done with an external service like fuzzing. i.e. Running the function with generated parameters.

seancorfield07:05:33

Well... yes... I mean, you can write any predicates you want, including ones that depend on external services... I wouldn't recommend it, tho'... doing generative testing against that is... fraught.

seancorfield07:05:25

Note that the section you're referring to explicitly shows how to stub the external service so you don't actually call it.

seancorfield07:05:41

"And then we want to test the behavior of run-query while stubbing out invoke-service with instrument so that the remote service is not invoked:"

seancorfield07:05:21

"The first call here instruments and stubs invoke-service. The second and third calls demonstrate that calls to invoke-service now return generated results (rather than hitting a service)."

seancorfield07:05:34

So... don't call external services in a spec...

xtreak2907:05:36

Got it. Thanks.

montanonic08:05:51

I'm new to Specter and trying to figure out how to express finding all of the numbers in deeply nested data, except for those that are nested within maps (at any depth) containing a particular value.

montanonic08:05:36

Finding all the numbers is totally easy, (traverse (walker number?) data), it's the filtering out part that I'm struggling with.

seancorfield08:05:39

Maybe #specter ?

👍 4
lepistane15:05:02

hi i am editing someone elses project and they are using environ (https://github.com/weavejester/environ) but it isn't in project.clj as dependency it's just called in ns like environ.core :refer env and it works?! how is this possible?

manutter5115:05:01

Most likely there’s a dependency that’s pulling it in. Try looking through the output from lein deps :tree

lepistane15:05:10

yep that's true

lepistane15:05:15

didn't even though of that

lepistane15:05:43

wow implicit dependency first time to come face to face with this thank you very much

dpsutton15:05:04

for sure add it to project.clj with the same version that comes in

☝️ 4
pavani20:05:15

Is there a way to refer to a map key within a map. Something like this:

(def map1 {:aaaa 1
                   :bbbb 2 
                   :c (* (:aaaa map1) (:bbbb map2))}

noisesmith20:05:06

not within a single form, no

pavani20:05:46

I'm trying to do this in config files without duplicates

noisesmith20:05:17

you'll need something other than a default clojure reader, or a preprocessor, in that case

Alex Miller (Clojure team)20:05:50

There are some libs to do this

Alex Miller (Clojure team)21:05:13

Crap, can’t remember what it’s called

pavani21:05:07

This is a pretty good lib for my requirement but it's like using a whole lib for just one small action. Thought i'll find out from the clojure PROS before I go with this lib

pavani21:05:48

Thankyou! Looking into this

pavani21:05:16

Thank you! looking into this too.

Alex Miller (Clojure team)21:05:03

ah, fern is what I was thinking of

montanonic21:05:11

I have a large set of data where the elements have the following form {:from _ :to _ :value _} For every :from :to pairing, there exists the opposite, so if there's {:from "a" :to "b"} there will also be {:from "b" :to "a"} somewhere in the collection. The :values in such maps will not be duplicates. I am trying to think of an elegant way to, for every point of data {:from :something}, look up its pairing, which will contain the key {:to :something}, and more specifically, get both of the values for those two pairings.

montanonic21:05:54

It is some kind of data join, but set/join is either not the tool for this, or I'm using it wrong (likely).

mg21:05:20

perhaps something like:

(reduce (fn [values {:keys [to from value]}] (update values (set to from) conj value) {} your-data)

mg21:05:43

that will give you a map where the keys are sets of the to and from, and the values are vectors of the values

noisesmith21:05:46

sounds like an adjacency list made of a map of node to set of connections with target and value could help

noisesmith21:05:33

@mg that loses information about direction, but hey maybe that's OK

👍 4
mg21:05:17

if direction information is important, you can conj in the whole element

mg21:05:35

the form of the question made it sound like it wasn’t, but nbd either way

montanonic21:05:39

Direction is ultimately not important here, the result will be a set of maps of the form {#{from to} (+ amount1 amount2)}, where from and to are from the pairing, and amount1 and amount2 are their two values.

mg21:05:47

oh well then in that case the reduce function could be (fn [values {:keys [to from value]}] (update values (set to from) (fnil + 0) value))

👍 8
montanonic21:05:39

minor typo: (set [to from])

montanonic21:05:59

I'm trying it out, thanks so much for the feedback

montanonic21:05:04

I didn't consider using update

mg21:05:16

my bad, the way I was using it should have been hash-set

montanonic21:05:43

@noisesmith when say adjacency list, what would be the nodes?

montanonic21:05:08

I'm not very familiar with working with graphs but I would like to learn more, and I'm not clear on what the nodes would be

noisesmith21:05:37

the from and to are each a node

👍 4
noisesmith21:05:02

and the map you start with is an edge between them (with a direction)

noisesmith21:05:23

once you have adjacency list format, there are some nice algorithms that all just work

noisesmith21:05:35

(if you need to do graph stuff)

noisesmith21:05:26

in eg. c you would just use a tree with pointers that might loop back, in a language that uses immutable data an adjacency list is much nicer

montanonic21:05:28

okay, very cool. I feel like I'd want to find some problems that require graphs to solve to learn more; seems fun and useful

noisesmith21:05:49

also with adj-list, walk over all nodes is practically free :D

👍 4
montanonic21:05:30

@michael.gaare also, where you using fnil as a shorthand for partial in this, or does it actually do something different here?

montanonic21:05:47

I've never used fnil before

mg21:05:47

It does something different

dpsutton22:05:09

(update {} :missing-key (fnil + 0) 17) -> {:missing-key 17}

montanonic22:05:39

@michael.gaare your solution works perfectly by the way 🙂, I'm just working on understanding it fully now

dpsutton22:05:03

fnil lets you say what to substitute if you get nil. so here you're updating a map at a key that won't be found. so the plus function would bomb, but you say, in that case, use 0 and carry on with the operation (here adding 17)

montanonic22:05:59

Ah okay, so @michael.gaare’s solution ensures that if for any datum an opposite pair didn't exist, nothing would explode

montanonic22:05:10

@dpsutton that totally makes sense, thank you

dpsutton22:05:34

because half the time you expect there to be no key

montanonic22:05:38

so basically, fnil gives a function default positional values

noisesmith22:05:26

right, most always people are just defaulting the first, but it accepts more

👍 4
dpsutton22:05:09

up to three values it appears. forgot about that. check (source fnil) @montanonic

montanonic22:05:06

yes, up to three, just tested

tdantas22:05:25

hey clojurians, I’m implementing my tcp server and I have one namespace which I called protocol protocol namespace will be responsible for encoding/decoding the bytes I’ve received from clients.

(ns tcp-protocol.protocol
  (:require [gloss.core :as gloss]
            [ :as io]))

; frame format
(def frame-format (sorted-map :opcode :byte, :payload (gloss/repeated :ubyte :prefix :ubyte)))
(def frame (gloss/compile-frame frame-format))

(defn encode [opcode message]
  (io/encode frame {:opcode opcode :payload (map identity message)}))

(defn decode [buffer]
  (io/decode frame buffer))
I’m not 100% happy because my encode/decode is using one global or singleton definition frame and I don’t want the callers have to pass the frame everytime how you guys would solve that ?

dpsutton22:05:49

overload the arity if you like

dpsutton22:05:00

(defn encode 
  ([opcode message]
   (io/encode frame {:opcode opcode :payload (map identity message)}))
  ([frame-format opcode message]
   (io/encode frame-format {:opcode opcode :payload (map identity message)})))
with a better name for frame-format?

dpsutton22:05:06

can also make the 2-arity call the 3-arity with frame passed as an argument although it would be the same with one line less repeated

tdantas22:05:33

yeah, that is how you would do it ?

tdantas22:05:56

I can’t see better way ( the frame is only protocol concern )

tdantas22:05:11

the problem is (([frame-format opcode message]) is almost dead code, no one will be using ( only on my tests )

dpsutton22:05:26

that's a good use of an arity in my book

dpsutton22:05:55

or you could make a function that takes a frame format and returns the encoder/decoder using that. then you could create that in tests. there are lots of ways

noisesmith22:05:43

another option is to have a regular function intended for others to call, and an implementation function - but that's effectively the same as the arity thing (some would consider one cleaner, some the other)

dpsutton22:05:14

I'm a big fan of that one too. i'm ok with def-ing the one way to use something in reality

tdantas22:05:36

interesting ! yeah > or you could make a function that takes a frame format and returns the encoder/decoder using that. then you could create that in tests. there are lots of ways

tdantas22:05:32

@noisesmith ( sorry to bother you ) > another option is to have a regular function intended for others to call, and an implementation function didn’t get that

noisesmith22:05:46

(defn foo*
  [impl x]
  ...)

(defn foo
  [x]
  (foo* (make-impl) x))

noisesmith22:05:02

so basically the same thing you would do with arities, mainly useful if you also have a use for arity dispatch

tdantas22:05:39

yeah, thank you guys !

tdantas22:05:18

@dpsutton btw, what is your book, gonna buy now 😄

dpsutton22:05:42

i recommend a book by @alexmiller called https://pragprog.com/book/shcloj3/programming-clojure-third-edition. Its got a good mix of java interop and real world practical things that I think other books don't emphasize enough

dpsutton22:05:55

I'm not a writer. just like chatting with people new to clojure

tdantas22:05:31

cool, thank you for the tips and book advice

tdantas22:05:35

cheers guys

👍 4
jeremy22:05:08

What’s a good book for going beyond the basics?

jeremy22:05:34

(I didn’t realize a book was just posted above too… Hah) 🙂

noisesmith22:05:48

Clojure Applied and Joy of Clojure both go beyond basics in their own interesting ways

noisesmith22:05:06

applied going into the little details of using it in production and non-obvious tricks, JoC going into some more theoretical and design level insights about the language

montanonic23:05:46

How do people feel about when to use reduce over loop recur? By making the accumulator for reduce a vector you can hold onto multiple pieces of state just like loop. Is there any semantic difference between the two at that point? Especially since you can short-circuit via reduced?

montanonic23:05:37

oh, well, wait, duh, reduce handles the sequential navigation through a data structure for you, while loop/recur does not.

noisesmith23:05:09

Always use reduce instead of loop if you consume the input in order. It can even return early with reduced.

noisesmith23:05:50

and yes, using a collection as an accumulator in order to track multiple values is normal

montanonic23:05:28

awesome, cool; glad to know what I'm doing is idiomatic

noisesmith23:05:07

there are other more specialized functions which can be better than reduce as well, but that's more case by case specific

montanonic23:05:14

right, totally

noisesmith23:05:18

see also transduce if you need speed and/or need pre-processing of the input like mapping or filtering or partitioning etc.

noisesmith23:05:34

but that's likely outside the beginners topic