Fork me on GitHub
#clojure
<
2017-09-07
>
qqq00:09:05

when destructuring via

(let [{:keys [a b c]} obj] ...)
is there a way to provide default values for a, b, c (ir they would otherwise be nil)

Alex Miller (Clojure team)00:09:06

yep: (let [{:keys [a b c] :or {a 1, b 2, c 3}} obj] ...)

Alex Miller (Clojure team)00:09:56

there has been some confusion (and bugginess) re the :or map in the past - just remember that the keys are always the local symbols being created. I find that helps resolve most questions.

dominicm07:09:12

Realising this the other day resulted in me getting a recursive restructure to work. Felt epic.

noisesmith00:09:38

list subtraction as a transducer:

(defn list-
  ([targets]
   (let [victims (volatile! targets)]
     (fn ([rf]
          (fn
            ([] (rf))
            ([result]
             (rf result))
            ([result input]
             (if (and (seq @victims)
                      (= input (first @victims)))
               (do (vswap! victims rest)
                   result)
               (rf result input)))))))))
user=> (into [] (list- [1 2]) [1 1 2 3 4])
[1 3 4]

qqq00:09:57

@alexmiller : nice, thanks! ( I searched for 'default' on that page, but only got:

(let [{category :category, :or {category "Category not found"}} client]
  (println category))
which is less elegant than [:keys [category]} as in your example good to know that :or also works with {:keys [...]}

Alex Miller (Clojure team)01:09:07

feel free to send a PR on the guide if you can make it better! https://clojure.org/community/contributing_site

theeternalpulse01:09:00

How much memory should the java process of the running repl take up in general on my personal computer I get this, but on my work mac it's a fraction of the memory cost

theeternalpulse01:09:25

personal computer is using the Openjdk

seancorfield02:09:37

@theeternalpulse It depends on a combination of things (including the default heap size on each platform) -- but most importantly, the JVM will happily use as much memory as it is "allowed" (by the system) and doesn't necessarily give it back to the system, just because it's done GC and isn't using as much.

seancorfield02:09:05

This is pretty normal for JVM stuff. Is Clojure your first exposure to the JVM?

seancorfield02:09:32

(where "allowed" is determined by a variety of factors)

theeternalpulse02:09:25

Yeah, not since college or menial work tasks have I really been exposed or the least bit interested in the JVM.

seancorfield02:09:50

Right now my REPL on a Mac is using just over 400MB, and I often have Java processes using 1-2GB locally. Our production Java processes run from a few GB up to 10-15GB.

theeternalpulse02:09:54

and the repl adds that two java tasks in the loop which is different than I'm used to

seancorfield02:09:04

That's Leiningen, not the REPL.

seancorfield02:09:40

Leiningen starts a process for itself and then starts a new process with whatever configuration comes from project.clj.

seancorfield02:09:56

Boot uses a single JVM process. A bare Clojure REPL is a single JVM process.

theeternalpulse02:09:30

interesting, I haven't used much of boot

seancorfield02:09:56

We switched from Leiningen to Boot late 2016 and we've been very happy with that change.

seancorfield02:09:00

It allows you to easily load new dependencies into a running REPL (with Leiningen, you have to restart your REPL), and you can easily extend Boot by writing new "tasks" as Clojure functions that compose into a "task pipeline".

theeternalpulse02:09:12

ah, lot of references still in lein, but I guess it's easy enough to switch or support both.

seancorfield02:09:34

Leiningen is the older build tool and therefore much more widely used, but I think Boot is gaining in popularity.

theeternalpulse02:09:08

thanks for the clarification though

seancorfield02:09:42

What I like about Boot is how easy it makes it to explore a new library in a REPL, outside a project: boot -d group/artifact repl and it fetches the latest version of that library, adds it to the classpath, and starts a REPL so you can play with the library interactively.

theeternalpulse02:09:44

that's pretty cool

seancorfield02:09:54

(also, Boot supports both Leiningen templates and Boot templates for creating new projects -- you'll generally still get a Leiningen-based project from a Leiningen template tho'...)

seancorfield02:09:08

There are channels for both #leiningen and #boot here if you want to dig deeper into either of them. Also #beginners is a good place to ask new-to-Clojure questions since a lot of helpful folks have opted into that channel to deal with all those early learning curve things!

andras.liter07:09:07

Hi guys. I am having a hard time creating a simple unit test using with-redefs for mocking on Windows. The test code, the function I'm about to test and the function to be mocked are all in different namespaces/files. I think I use with-redefs correctly, my unit test runs fine on Linux machines (docker container or virtual Ubuntu), but it just does not work when executing/developing them on my Windows machine: the with-redefs bindings are not applied and the test wants to make real e.g. http calls, which I'd like to mock. It is the same case if I run lein test, or try executing the test from a REPL through Eclipse CCW. Do you have any idea what I'm doing wrong? Thanks

schmee07:09:27

with-redefs is full of quirks and edge cases where it doesn’t work

schmee07:09:48

I’ve had success using protocols and https://github.com/metametadata/clj-fakes for mocking

andras.liter07:09:57

Thanks @schmee, yes I've read that it does not work well with async, but did not expect to encounter such platform specific differences. I'll give clf-fakes a try

rohitverma10:09:48

hi all, I have just joined the group, I am seeking some help in creating a project of app marketplace, which basically allow to add remote extensions to our current app

rohitverma10:09:12

can anyone suggest me good guideline/ software architecture pattern for that

nooga11:09:36

what’s the best way to try things out in clojure? I’ve been using lein and cider for years now but sometimes I just want to quickly play with something in a single clj file/buffer, without setting up a project.

jumar11:09:34

You can leave it open with REPL running most of the time and whenever you have something to try, you can quickly test it. You also have the convenience of Cider+Emacs and you can save your experiments (just in case you want to revisit them in the future).

jumar11:09:53

I was inspired by Stuart Halloway's talk on Repl Driven Development: https://vimeo.com/223309989#t=33m47s

nooga12:09:09

thanks! @U06BE1L6T I’ve had similar scratch projects in the past but I usually forget about them and create new every time I want something so I thought that maybe there’s a more lightweight solution out there 😉

jumar12:09:48

other than that I just have a bash alias clojure_repl aliased to rlwrap java -jar /Users/jumar/.m2/repository/org/clojure/clojure/1.8.0/clojure-1.8.0.jar. But I almost never use it (mostly only if I try to make sure that leiningen is not interfering with the functionality I'm trying to test)

seancorfield13:09:03

Boot lets you start a REPL with any dependencies you want, outside a project. Maybe that's worth looking at? (I'm a big fan of Boot, having switched from Leiningen at the end of last year!).

nooga14:09:35

Thanks @U04V70XH6, can it be used with cider so that I get to edit my code in emacs and run it bit by bit instead of typing directly into the repl?

nooga14:09:09

will check it out, thanks!

miro-impressa13:09:12

Hey guys. I would like to us migratus as a plugin with lein and postgres. I was going through the documentation provided and found the initial setup but I would like to use one DB config somewhere in settings and just pull it to project.clj. I am new to clj and I am not sure if its even possible. thx

fmind14:09:56

@nooga you can use lein-try for testing a single lib, or use ~/.lein/profiles.clj with lein with-profile [PROFILE] repl

nooga14:09:08

thanks @fmind, I need to check it out 🙂

pesterhazy14:09:01

(cond
  (re-matches #"^/product/(.*)$" url)
  (let [product (second (re-matches #"^/product/(.*)$" url))] ;; regex repeated
    (do-something-with-product product))
  (re-matches #"^/category/(.*)$" url)
  (let [product (second (re-matches #"^/category/(.*)$" url))] ;; regex repeated
    (do-something-with-category category))
  ;; more of the same
  )

pesterhazy14:09:20

This is repetitive and error prone. What's the best option if I don't want to repeat the regex for each branch here? I know I can use a nested if-let, but that doesn't scale and isn't readable.

ghadi14:09:38

@pesterhazy one simple option: instead of doing the dispatch and handling in the same block, just do dispatch:

(cond
  (re-matches #"^/product/(.*)$" url)
  :product
 ...

ghadi15:09:02

...then use the val to dispatch through a multimethod

pesterhazy15:09:41

@ghadi but that doesn't give me access to the match groups

ghadi15:09:03

precisely.

ghadi15:09:14

do that in the individual multimethods

ghadi15:09:10

if you are seeking clarity and not being error-prone, consider separating the branching from the handling.

ghadi15:09:59

if your regexes are similar, you could even dispatch a multimethod with it:

#"^/(product|category)..."

pesterhazy15:09:02

@ghadi but I do need to match the regex twice If I understand correctly right?

ghadi15:09:31

another option is to write a macro that makes the nested if-let look less gross

misha15:09:42

is wrapping all of it in a let with 2 re-matches bound too expensive for you?

ghadi15:09:49

IMHO go for clarity first. then if you need performance, set a budget. only change code iff you are over the budget.

ghadi15:09:40

i know you mean two per branch, but you're already matching regexes much more than twice

ghadi15:09:25

stuart sierra has a macro called nested somewhere, but I can't find it

pesterhazy15:09:53

yeah something like a cond-let really seems called for

ghadi15:09:35

you can certainly use that

ghadi15:09:50

i think you were hinting at wanting to use that in your original question, but I wanted to offer an alternative

misha15:09:43

(condp re-matches "/product/foo"
  #"^/category/(.*)$" :>> #(->> [:category (second %)])
  #"^/product/(.*)$"  :>> #(->> [:product (second %)]))
;; => [:product "foo"]

pesterhazy15:09:20

@misha, that's an option I hadn't considered,thanks!

burganov15:09:46

what is the best technique for consumer/producer pattern? I think about standard java concurrent collections and core.async. What pros and cons about these two? Are there other more good desicions?

tbaldridge15:09:55

java concurrent collections don't really do synchronization (unless you use queues). Java queues don't support callbacks or lightweight threads, and that's why core.async was created.

burganov15:09:40

I think about exactly queues, ConcurrentLinkedQueue for instance

noisesmith17:09:01

core.async simplifies many cases for more complex coordination between multiple queues (channels) operating in a coroutine / callback based setup that allows efficiently sharing a small pool of threads with a syntactic transformation that turns something that looks like normal procedural code to implicitly use callbacks (go blocks) plus a separate expandable thread pool for longer running / blocking tasks (async/thread)

noisesmith17:09:31

you could do the same work with two thread pools and a bunch of queues and a whole lot of callbacks, but the result would not look nearly as nice (and would be error prone)

heptahedron17:09:09

I can't seem to get current namespace auto-resolved keyword map syntax to work in clojure-1.9.0-alpha19 to work, is it not present in this version? Something as simple as #::{:foo :bar} is getting me an unmatched delimiter error

hiredman17:09:14

it will depend on your repl too

heptahedron17:09:23

Oh, I didn't think about that

hiredman17:09:01

it definitely works in the clojure.main repl with clojure-1.9.0-alpha19

heptahedron17:09:09

I'm getting that too, thanks for the help!

ghadi18:09:12

Anyone here have experience with JSON-LD?

kah0ona19:09:07

Question about My First Transducer: https://gist.github.com/Kah0ona/789d2daf0cae1a9773f0a652272723b5 I basically want to create a transducer that, if two consecutive maps in a collection have the same :type, that they be merged into one according to some merge-fn

kah0ona19:09:40

but my reasoning for the actual merging is wrong I think, because i cannot use ‘pop’ and ‘conj’ to replace the last element in the ‘result’ var

kah0ona19:09:43

(I reckon)

noisesmith19:09:54

@kah0ona wouldn’t [prev {:type ::none}] make sense? because you wouldn’t want to accidentally merge a map with no type key to the keyword none right?

noisesmith19:09:10

alternatively, you could store the type of the last thing, and not the map itself

kah0ona19:09:16

good point

kah0ona19:09:52

but can this work at all? Specifically worried about the meat of the thing:

(-> result
                 pop ;; <-- !!! pop and conj are too specific functions that deal on collections. 
                     ;; What should I do to make this work?
                 (conj (merge-peak-with-first prior input)))

kah0ona19:09:02

can I use pop and conj on ‘result’ at all?

noisesmith19:09:13

your caller should be managing the input as a series of things

noisesmith19:09:20

your transducer can’t be the one to manage that at all

noisesmith19:09:30

because the input could be a channel instead of a collection

noisesmith19:09:38

or simply a set (which can’t be popped)

kah0ona19:09:40

that is what is worrying me 🙂 I see that

kah0ona19:09:49

but then, how would that be refactored

noisesmith19:09:01

look at how distinct is defined

kah0ona19:09:01

is it transducable at all, this concept i’m trying to do?

kah0ona19:09:14

aah thanks, ok.

noisesmith19:09:24

a similar problem just a longer memory than you have, and a simpler substitution (output, no output)

kah0ona19:09:28

i was already looking into some core transducers to get some inspiration

noisesmith19:09:57

yeah, distinct is probably the simplest one that has the individual moving pieces you need (sometimes not creating an output for a given input)

noisesmith19:09:21

plus carrying its own state

kah0ona19:09:58

okay, i’ll look into that, seems logical.

kah0ona19:09:29

thanks a bunch for the swift and adequate pointers 🙂

noisesmith19:09:26

adequacy is the standard I strive for 😄

kah0ona19:09:46

when this transducer works, i’ll print it and make it a poster to hang in my bed room 😉

kah0ona19:09:05

as it is My First

kah0ona19:09:15

I’ve used them before, never created one myself

kah0ona19:09:22

them = the core ones

noisesmith21:09:51

I have a hunch there’s a way to avoid needing to check for ::none as a type everywhere, but if that’s unavoidable you could also make a small helper something like (defn empty-state? [cp] (= ::none (:type @cp)))

noisesmith21:09:35

or maybe you always want to bind outside the function, but it might be clearer with that

noisesmith21:09:03

in fact, if you defined it in the let block on line 4, you could make it a no-arg function

kah0ona21:09:29

yeah good improvements

kah0ona21:09:56

i’ll tweak this thing once I finished my champagne in celebration of my first XF

kah0ona21:09:11

that’ll be first thing tomorrow

misha21:09:50

Volatiles are faster than atoms but give up atomicity guarantees so should only be used with thread isolation
how would an example of "thread isolation" look like?

sundarj21:09:28

i could be wrong, but doesn't that just mean that you shouldn't share volatiles between threads?

leonoel07:09:41

@U61HA86AG volatiles are all about multithreading. there is really no point of using a volatile if it isn't ever shared between threads.

leonoel07:09:28

@misha I wish thread isolation would be better explained in the documentation. The key point is that atoms are able to manage concurrent updates and volatiles are not. That means, if you make it possible for two threads to vswap! the same volatile at the same time, update consistency will eventually break. To prevent this, updater threads must be synchronized by another mean (e.g locks).

leonoel07:09:16

So this is my understanding of thread isolation. It can be always in the same thread, or it can be multiple threads using synchronization to prevent concurrent updates.

leonoel08:09:36

for example go blocks are thread isolated, so it's safe to vswap! the same volatile multiple times within a go block, even though updates might be run by different threads

sundarj08:09:24

ah, interesting. but in the case of a transducer, that volatile is private, so how could it ever be shared?

sundarj08:09:19

also sharing between threads without any concurrency guarantees (unless you use another mechanism) is what i meant by 'shouldn't share'

sundarj08:09:38

to my mind, volatiles are vars with atoms' API

leonoel08:09:41

vars and volatiles differ significantly in usage and implementation. static vars are mutated via locks, dynamic vars use ThreadLocal, and both are generally used as top-level namespace items. volatiles are almost always used locally and have no means of update synchronization.

sundarj08:09:43

ah, right. thanks for the info 🙂

leonoel08:09:43

the transducer case is interesting. as you said, state is private (one could say encapsulated but it's kind of blasphemous in clojure spheres), and the transducer itself doesn't implement any synchronization mechanism because it's up to the transducing context to ensure sequentiality (it may involve multiple threads, e.g for core.async channels).

leonoel09:09:09

however, the question of why stateful transducers use volatile instead of bare unsynchronized variables is still pending https://dev.clojure.org/jira/browse/CLJ-2146

sundarj09:09:09

i see. very intriguing.

misha14:09:54

@U053XQP4S thank you for the explanation. I had a question on what happens when stateful transduser is used in parallel, or how could it be used. But I had (and still have) a hard time framing the question opieop

leonoel14:09:55

what is your use case ?

misha14:09:17

that's the "problem" – I don't have one :). Was just trying to figure out why/when should I use volatiles, as those came up few messages above in a transducers context, and, as far as I read on the internet, – were actually introduced because of transducers

leonoel14:09:11

that's right, stateful transducers imply use of mutable state so there was a need for a lightweight mutable container

leonoel14:09:59

however I find the decision to make it volatile quite confusing

leonoel14:09:23

because it has memory visibility and performance implications that are not very clear

leonoel14:09:08

about parallelism, I can't really imagine a situation where a stateful transducer could be parallelizable

misha14:09:32

so things like partition-by are out of question, right?

leonoel14:09:48

parallelism as in fold ?

misha14:09:18

if so, why not use transient? Is it because only basic collections can be transient, and anything a bit nested would be pain to use?

leonoel14:09:21

or as in core.async pipeline ?

misha14:09:09

I think fold, because I did not use pipeline yet, don't know what that actually looks like

leonoel14:09:31

it's perfectly OK to use transients in stateful transducers, but you still need a mutable container

leonoel14:09:29

transient aren't identities like java mutable collections

leonoel14:09:48

pipeline and fold can't be used with stateful transducers, it has been made pretty clear in RH talks about transducers

misha14:09:56

I need to watch it again then. Missing so much information when listening without actual question/use case

madstap21:09:44

@kah0ona You nerd-sniped me with that problem. Here's my take on it: https://gist.github.com/madstap/a7d158ef0c3e7b5bbf5cd55c5de4c913

noisesmith21:09:58

why change the volatile into an atom?

madstap21:09:30

I made my version from scratch, just an oversight. Should be volatile!, vswap! etc.

ghadi21:09:10

If you want to play on Hard Mode, write some generative tests on it

ghadi21:09:46

Last time I did that for a transducer, I found a nasty bug with reduced

seancorfield22:09:41

@alexmiller Is the new ## reader macro extensible in any way? Or is it just intended for built-in symbolic values?

seancorfield03:09:34

Thanks. I figured as much, but just wanted to check.