Fork me on GitHub
#clojure
<
2017-04-20
>
tech_hutch02:04:45

Quick question

tech_hutch02:04:50

If I require a namespace in one file, and change one of the vars in that namespace, will that var be changed or have its default value when I require the same namespace in another file?

tech_hutch02:04:53

(Maybe should have posted this in #beginners)

tbaldridge03:04:40

@tech_hutch no it won't change, you have to re-evaluate the var you changed

tbaldridge03:04:50

or re-load the file (via an editor or a call to load-file

tech_hutch03:04:08

I don't understand

tbaldridge03:04:03

well, what do you mean by 'change one of the vars in that namespace'?

tbaldridge03:04:07

how are you changing it?

tech_hutch03:04:29

So, if I require namespace b from namespace a, change b/thing to "second value", then require b in namepace c, what will b/thing be?

tbaldridge03:04:41

how are you changing it?

tech_hutch03:04:11

Uh, set!, I suppose. I hadn't really thought about it yet.

tbaldridge03:04:40

there is only one version of a var with a given name, so foo/bar will be the same in every namespace that requires it.

tech_hutch03:04:42

So changing it in one file will change it for another? (I can guarantee that c will run later than a.)

qqq03:04:06

is there a one liner for doing gunzip in java ?

qqq03:04:28

is there a one liner for doing gunzip in clojure, without sending it to the shell to call gunzip? i.e. using pure java libraries

noisesmith03:04:23

@qqq java.util.zip.GZIPInputStream should be an option if nothing else

noisesmith03:04:49

@qqq

+user=> (slurp "h.gz")
"�\b(�X�H���� 0:6"
+user=> (require '[ :as io])
nil
+user=> (-> "h.gz" (io/input-stream) (java.util.zip.GZIPInputStream.) (slurp))
"hello\n"

noisesmith03:04:55

that's a decent one liner I think

noisesmith03:04:58

@qqq if it helps, I just googled "gzip java" looked at the first page with a blog post with some code, squinted my eyes and looked for a class name with "GZIP" in the name and ignored everything else, then proceeded to try a few things in the repl

noisesmith03:04:42

(of course this assumes a bunch of past experience using interop and java io stuff, but hopefully the process is encouraging)

noisesmith03:04:33

my next step would have been to look at the javadoc, but my first guess (that something with InputStream in the name needs another InputStream as its constructor arg) worked out

tbaldridge03:04:14

@tech_hutch you're kindof confusing editing files with set!. Editing files does nothing unless you load them or run them in a repl.

tech_hutch03:04:05

I'm not talking about editing files

tech_hutch03:04:18

How would I modify a var from a namespace I'm requiring? def?

noisesmith03:04:25

every namespace sees the same values of vars, including any mutations

noisesmith03:04:49

@tech_hutch don't - there's things that actually are designed for programmatic mutating, only change defs as a part of a dev workflow

noisesmith03:04:21

if someone else needs to change it it should hold an atom, ref, or agent that gets dereffed on usage

tech_hutch03:04:36

I may not be using the right terminology

tech_hutch03:04:44

In namespace b, let's say I have (def *thing* "text"). I want to be able to set this from another file. What's the best way to go about this?

noisesmith03:04:15

that's not what vars are for

noisesmith03:04:03

if you need to mutate a value in namespace b, define it as a proper mutable type (def thing (atom "text"))

noisesmith03:04:57

then you can just do eg. (reset! b/thing "other text")

noisesmith03:04:15

but even better is to avoid needing to mutate top level values, but that's a whole other conversation

didibus06:04:21

tech_hutch:

(def ^:dynamic *my-dynamic-var*)

(binding  [*my-dynamic-var* 10]
  *my-dynamic-var*)

didibus06:04:21

Binding rebinds the value of the var but only between its s-expression.

didibus06:04:55

And the guide says to surround dynamic vars meant to be rebinded with earmuphs

noisesmith03:04:04

it's talking about dynamic vars, thread local bindings

didibus06:04:32

Is there anything like danielz/system but for mount? I guess just a set of ready made defstates?

qqq07:04:49

in java.nio.ByteBuffer , how do I set the value of "arrayOffset" or the "posiiton" field ?

qqq07:04:03

it's the field things like get, getInt, .... read from and increment

qqq07:04:19

I forgot to look at the superclass. Thanks! 🙂

leonoel07:04:19

it's easy to forget inheritance is a thing after doing a bit of clojure 🙂

yonatanel09:04:37

@didibus Would you use it as a dependency or as a copy-paste repository of examples?

didibus16:04:10

yonatanel: I was thinking dependency, but a repository of examples would be a good start

pyr09:04:28

Hi #clojure

pyr09:04:42

I wonder if I'm the only one to often need something like this:

pyr09:04:59

or if there is a better way to do that

pyr09:04:47

(I mean, besisdes (def realized-chan #(doto (a/promise-chan) (a/put! %)))

rauh10:04:16

@pyr Couldn't you just (go val)?

pyr10:04:13

I was afraid this might result in too much, but yes, visually that's much better.

leonoel10:04:17

if you want to be able to take multiple times, and always get the same value, (go val) won't work

pyr10:04:30

@rauh thanks for the sanity check

pyr10:04:44

@leonoel el nope, taking once is what i'm going for

leonoel10:04:17

looks good then

viesti11:04:49

has anyone done custom decoding with cheshire?

viesti11:04:25

would like to parse timestamps

viesti11:04:27

since cheshire already uses transients, was thinking that it would be neat to do transform while accumulating the transient

tjtolton12:04:26

transients? Where do I go to learn more about this @U06QSF3BK ? I'm currently getting burned by trying to convert dashed clojure keywords into underscored keys in the json response from a web api. It's too slow the way I'm doing it, but if I can't figure out how to do it performantly, we'll be forced to use underscore keys in our clojure code :c

viesti12:04:45

(parse-string stuff (fn [k] (keyword (.replaceAll k “-” “_") )))

viesti12:04:06

or something like that, parse-string takes a function to convert keys

viesti12:04:11

parse-object and parse-array use transient within cheshire: https://github.com/dakrone/cheshire/blob/master/src/cheshire/parse.clj#L19

viesti12:04:37

was hoping to have place where to convert values too, within the loop that modifies the transient

viesti12:04:34

key-fn is applied within the loop that modifies the transient that accumulates a json object, so this would probably work for you

tjtolton12:04:53

yeah, that's the level I set the transformation at -- :key-fn #(-> % (str/replace #"-" "_") keyword).. given what you've told me, perhaps my bottleneck is elsewhere

viesti11:04:04

could do as a post process step but got attracted by speed 🙂

qqq11:04:46

how do I print a float-array ?

danielsz11:04:13

@didibus danielsz/system has an experimental branch where mount is used instead of component.

danielsz11:04:07

Feel free to use it, just be aware it's not released and you're on your own.

qqq11:04:51

(print (seq ... float-array ...))

dimovich11:04:57

hello fellow clojurians

dimovich11:04:06

I'm writing a selenium application, and would like to be able to pause the application from the console

dimovich11:04:40

what is the best approach to listen to input from user on console and in parallel running the selenium tests?

Alex Miller (Clojure team)12:04:05

@pyr I think I would use to-chan for this with a vector of one element

pyr12:04:00

hadn't think of that either

pyr12:04:06

*thought

dimovich14:04:22

this solved my blocking console input needs

qqq16:04:13

why does clojure's let default to SEQUENTIAL bind instead of PARALLEL bind? one advantage of parallel bind is that when readin the code, it says: we don't guaratnee the order or execution -- so when I'm reading line 3 of the let, I don't ahve to wonder "did lines 1 & 2 do anything"

tbaldridge16:04:45

parallel bind with a let doesn't make a whole lot of sense

qqq16:04:00

(context: reading "out of the tarpit" // talking about the compelxity induced by order)

qqq16:04:10

@tbaldridge : ha! I was hoping you would answer 🙂

tbaldridge16:04:53

@qqq yeah, that's kindof what I was talking about the other day. Compare Prolog to Clojure and you can see the difference. Prolog doesn't have a defined order (most of the time), but you pay a bit in performance for that freedom.

qqq16:04:27

@tbaldridge : ah, I thought it was due to backtracking / search -- what does it have to do with order ?

tbaldridge16:04:22

Well if you don't have a backtracking execution engine, then most likely you have an imperative execution engine. In the case of an imperative execution engine (like the JVM) then a sequential let makes the most sense.

qqq16:04:07

emacs has let vs let*; clojure seems to default to let*

qqq16:04:23

What is true in the case of the JVM that is false in the case of Emacs, or did Emacs get it wrong?

tbaldridge16:04:31

Really I think it's this: sequential maps very cleanly to both imperative execution engines and stack machines. The JVM is both of those.

didibus16:04:14

@qqq I'm not sure about the implementation benefits, but I very often rely on the sequential nature of it. I tend to use a let instead of the threading macros most of the time because they are easier to read since they embed a name for each step and you can insert prints and logs and other cross-cutting concerns in-between the steps.

qqq16:04:48

I've found I used (as-> ... $ ) alot more

qqq16:04:55

precisely to avoid names like state0 state1 state2 state3 ...

tbaldridge17:04:08

um...you don't need that, just reuse the same name

tbaldridge17:04:26

(let [a 4 a (inc a) a)

qqq17:04:30

isn't shadowing bad ?

tbaldridge17:04:41

why would it be bad?

tbaldridge17:04:22

it's usage of lexical scope, it might be bad in languages that don't have proper scoping, but with let in clojure it works quite well.

didibus17:04:34

(let [names (get-names client :from x)
_ (println names)
sorted-names (sort names)
normalized-sorted-names (map normalize sorted-names)
_ (log "Names fetched and normalized to " normalized-sorted-names)
normalized-sorted-names)

qqq17:04:09

yeah, I'd prefer

(as-> .... $
  (sort $)
  (normalize $)
  ...))

didibus17:04:34

How would you print in between?

qqq17:04:47

(do (println $) $)

didibus17:04:52

Hum, ya, I guess it's just a preference of style. I work with a big team, so I find the naming helps others read my code, but I might try out the as-> a little more, see if I like it

tbaldridge17:04:00

well you're in a bit of a minority there. Using lets and shadowing is the most common style I've seen.

roberto17:04:13

I find the as-> macro hard to follow. Everytime I’ve used it and have to go back to reading what I wrote a couple weeks later, I can barely understand it.

tbaldridge17:04:15

(maybe because as-> didn't exist until about 2-3 years ago)

qqq17:04:27

I think I figured out why I think shadowing is bad

qqq17:04:06

Some other langauge (forgot which) had a linter, which flageed many things, of which one was shadowing.

qqq17:04:17

Ever since they, it was hard coded into my brain 'shadowing = bad'

roberto17:04:31

that is probably JS

qqq17:04:02

@roberto: I find -> and ->> impossible to read, but as-> ... $ to be quite clear

bronsa17:04:03

even in clojure shadowing of vars via locals is often a bad sign and can lead to bugs -- but shadowing of locals with other locals is very common

roberto17:04:14

I avoid shadowing in JS, but not in languages with proper lexical scoping.

didibus17:04:40

Ya, Clojure warns you if you shadow a core var

bronsa17:04:50

no it doesn't

roberto17:04:12

hmm, yeah, I think it does.

roberto17:04:32

It has warned me when I’ve created my own replace function

bronsa17:04:42

right, that's not locals shadowing tho

bronsa17:04:55

it's when you override a core var

didibus17:04:57

Well, something that my ide setups warns me about it in my repl then

roberto17:04:57

no, but it is a core var right?

roberto17:04:10

yeah, it is what @didibus said

qqq17:04:15

the other thing I like about (as-> ... $) over let is that: (1) with (as-> .... $), I know that "the flow is linear" (2) with let, the expr can be any directed acyclic graph

didibus17:04:29

Ya, I'm talking about core var shadowing only

roberto17:04:48

yeah, that is what I understood.

bronsa17:04:53

that is a warning that exists but it's a different thing than what we're talking about

didibus17:04:10

Ya, I'm all in for locals shadowing

didibus17:04:17

Do it all the time

qqq17:04:30

if you ever use Erlang, you'd be unplesantly surprised 🙂

didibus17:04:12

@qqq You mean like with let it's unclear it will just return the value of the last binding? But as-> makes that clear?

qqq17:04:45

no, as in if you draw the dependency of the exprs, with as->, it's a line and with let, it can be any directed acyclic graph

qqq17:04:16

(let [
a ....
b .... a ....
c ... a b ...
d ... c a ...
e .... d b ...
])

didibus17:04:43

Oh, but that's also why I tend to use let over the threading macros

bronsa17:04:22

@qqq so what? in real code that's often useful

didibus17:04:35

Cause I hate it when I'm threading and I realize oh crap, for this steps I actually need the value of 3 previous steps. With let I can do that no problem

qqq17:04:45

@bronsa: I'm not saying "kill let", I'm saying "if it's linear, I prefer to use as-> instead of let"

bronsa17:04:23

not many will agree with you that not naming intermediate values is better than naming them

qqq17:04:42

I don't thikn of it as "not naming intermediates", but as in "I have a pipeline"

qqq17:04:00

then I shove this object through the pipeline/functions, calling it "$" all the way through

didibus17:04:40

I can see myself using as-> instead of the times I use -> or ->>.

didibus17:04:23

@qqq to be fair, as-> feels a bit more functional and let a little more imperative, so good on you. But personally, the threading tend to confuse me. I even normally just nest s-expressions manually, I find even that to be clearer.

lvh17:04:01

my fav is when you use name in rubby and then change your mind and then it secretly becomes the name of the class

tbaldridge17:04:03

fun fact: as-> just compiles down to a bunch of nested lets that each shadow the one above it 😛

tbaldridge17:04:02

(as-> 1 a (+ a 4) (dec a)) is (let [a 1] (let [a (+ a 4)] (dec a)))

tjtolton17:04:08

so, im writing a custom :store function for ring wrap-multipart-params. are there any good examples of this out there (or any handy premades)?

cddr17:04:45

@tbaldridge You have advocated against with-redefs here in the past. You asked if there was some source recommending it's use for testing. Here is one such source http://blog.josephwilk.net/clojure/isolating-external-dependencies-in-clojure.html (albeit offering it as an alternative among others). It has come up once again in our dev team and even relatively experienced engineers are happy to recommend it's use (e.g. "it's ok in unit tests") I get the impression you wouldn't agree even with that. Would you mind rattling off a few of the pitfalls you've seen?

cddr17:04:26

I'll try to follow up with a doc and see if I can get it into http://clojure-doc.org/ or something.

tbaldridge17:04:26

@cddr, mostly around multiple threads. The problem is it's really hard to define when the scope of with-redefs terminates. The start of this blog post shows an example, then explains why this happens: http://blog.cognitect.com/blog/2016/9/15/works-on-my-machine-understanding-var-bindings-and-roots

tbaldridge17:04:24

Aside from the problem the blog post shows, it also makes it really hard to run multiple tests in parallel, as you can now have one test that changes global data for other tests.

josephwilk18:04:31

Yes, I mention that Its important to note that with-redefs are visible in all threads and it does not play well with concurrency.

josephwilk18:04:45

Its a while since I wrote that post and its very, very rare I reach for with-redefs. Tends to be a design smell that its required. Its really an edge case tool @cddr

cddr18:04:02

You're right joseph. Sorry for not highlighting that.

tbaldridge18:04:54

Yeah, and I can't think of a better way to say "doesn't play well with concurrency"...except it really doesn't play well. Like putting your kid in a cage with a lion, doesn't play well.

tbaldridge18:04:40

Even stuff like lazy seqs don't work: (with-redefs [foo 4] (map (fn [x] (+ foo x)) (range 10000)))

cddr18:04:41

One of the problems with "doesn't play well with concurrency" though is that sometimes people don't realize where the concurrency is in their programs.

cddr18:04:09

If one of the libraries we use handles all the difficult bits, it can be possible to "get shit done" without even worrying about concurrency

cddr18:04:51

I mean. I'm convinced. I just need to figure out how best to convince those on my team that remain seduced by the thing that seems easy until it isn't.

tbaldridge18:04:42

The other thing that might help, is you can explain that with-redefs mutates global state. Which goes against most of Clojure's designs. At least with binding the changes are local to the current thread.

cddr18:04:36

Yeah exactly. Thanks again.

josephwilk18:04:40

My general approach to convience people its a bad idea, is to explore how the code can be redesigned/reshaped to not need it. Then reflect on this code, is it easier to test, is it better designed. It usually is. And hence reaching for the with-redefs gets answered with, well lets first check this is not just bad design. It might be sometimes, though not all.

josephwilk18:04:05

I think the history of testing has us reaching for tools to fix hard to test code that really reflects our bad design.

roberto18:04:58

so, what is the alternative to redefs?

tbaldridge19:04:16

@roberto functional programing

tbaldridge19:04:12

Seriously that's it, pass these things in as parameters. If you need to mock something use Component/Integrant, etc. Or test the inputs to your side-effecting functions instead of the functions themselves

tbaldridge19:04:04

So for example if you need to test that you're sending a HTTP request correctly, don't stub out http-send, that's a IO function in the middle of your code, refactor to move http-send to the edges of your code, then write a "prepare-http-request" function and test the output of that.

roberto19:04:15

hmmm, yeah, but I find conflicting recommendations. For example, if I am building a web scraper with selenium, and I want to write some unit tests, I normally use redefs to mock interactions.

roberto19:04:16

Now if I follow the advice to use component or create protocols, I will only have one implementation (and a test implementation), but recently in the component channel the recommended advice by the creator of component was to not use protocols unless you have more than one implementation (and test/mock implementations don’t count).

roberto19:04:08

yeah, I get that type of unit testing, but at some point I want to test the interactions too

roberto19:04:18

which is what I use mocking for in Java/Kotlin

tbaldridge19:04:45

I don't know that I agree that "test/mock implementations don't count". Got a link to that recommendation?

roberto19:04:58

it was said in the slack channel

roberto19:04:04

the component slack channel

roberto19:04:31

unfortunately, it has been pruned from slack’s history

roberto19:04:29

but yeah, I find myself creating protocols when I’m in a bind trying to write a unit test. Especially if I want to figure out how high level abstractions will interact.

tbaldridge19:04:32

I agree that not everything needs to be a component, or have its own protocol, but anything you need to mock needs to be a component. That's pretty core to how component works.

tbaldridge19:04:40

So there might be a misunderstanding there.

roberto19:04:47

yeah, quite possibly

roberto19:04:12

thank you for the advice

dominicm19:04:02

A quick google couldn't find it, but someone who saw how it was worded could find it

g3ntleman20:04:02

Hey, folks! I'm still new to Clojure and wondered about the rational behind some functions returning lazy lists.

danburton20:04:17

(range) is an infinite list. Can't do that without lazy lists 😉

g3ntleman20:04:20

Especially, I stumbled over sort(vectr) returning a list.

g3ntleman20:04:18

It's very easy to sort a (copy of) a vector in place and return that.

danburton20:04:57

Not sure if clj sort takes advantage, but it only has to perform at most O(n) work to return the first element. So by leveraging laziness, you can in theory get better performance on the sort. Again, not sure if clj actually does this for this case.

g3ntleman20:04:59

Now, I'm getting a lazy list, which drops the O(1) random access feature.

tbaldridge20:04:18

@g3ntleman it's an easy extension of cons-cells in lisps. Assuming a cons cell is [head tail] and you can do [1 [2 [3 nil]]]. What if you could do [1 f] where f is a function, which when called returns the rest of the list?

danburton20:04:20

You can just call vec to dump it back into a vector

tbaldridge20:04:28

@g3ntleman bingo....lazy seqs

g3ntleman20:04:02

@danburton Yes, that's what I'm doing.

g3ntleman20:04:23

But that's probably more inefficient as it needs to be.

g3ntleman20:04:55

It constructs O(n) list cells that I don't need.

danburton20:04:08

If you are looking for down-to-the-metal absolute speed, then clj may not be for you. But you're dealing with a O(n) vector worth of memory anyways. O(n) cons cells aren't going to increase the memory big-oh footprint.

g3ntleman20:04:36

Other way to ask this question: Are there ways to sort a vector without constructing O(n) list cells?

g3ntleman20:04:52

(and get a vector back?)

danburton20:04:43

java interop, perhaps. But why worry about it?

g3ntleman20:04:49

Probably by falling back to java, right?

g3ntleman20:04:29

Additional O(n) garbage might make a difference.

danburton20:04:10

Is there a particular application domain where you are concerned about this difference?

g3ntleman20:04:17

So any idea, why it's handy that sort does not return the same collection type back that was put in?

jr20:04:10

vectors are persistent. there is no in place shuffling

g3ntleman20:04:56

We need to copy once.

danburton20:04:03

Most clj sequence operations return a lazy seq. This allows you to chain multiple transformations, and it only needs to pass over the sequence once.

g3ntleman20:04:06

But do we need to copy twice?

jr20:04:13

why do you need random access?

danburton20:04:48

So I guess clj trades memory for time

g3ntleman20:04:49

Because that's what my data structure is - a big table.

jr20:04:49

is there a reason you need it sorted and random access?

g3ntleman20:04:43

Yes, it's a sorted index.

g3ntleman20:04:08

"If coll is a Java array, it will be modified. To avoid this, sort a copy of the array"

g3ntleman20:04:58

So this should do it: (vec (sort (into-array (vector 3 2 1))))

g3ntleman20:04:19

sec does alias and sort does sort in place.

g3ntleman20:04:50

Nice that I can do that in Clojure without (explicit) interop.

g3ntleman20:04:07

Still not sure, why this is not the default behavior.

tbaldridge20:04:46

Probably because Java has a really fast sort algorithm that takes arrays as inputs. So the fastest approach is to convert to an array, sort, then call seq on the array which is O(1)

csm20:04:06

sort already converts the coll to an array though?

tbaldridge20:04:08

You'd have to pay O(n) at the end if you wanted the same type out @g3ntleman

csm20:04:24

when in doubt, read the clojure source

g3ntleman20:04:49

Clever. As expected.

uwo20:04:44

should i be wary of resolving vars at runtime, say in an endpoint that sees some traffic. like is there a reflection cost?

tbaldridge20:04:57

Example code? How are you resolving them?

uwo20:04:44

@(resolve 'my-proj/var-name)

noisesmith20:04:07

that’s eliminating the compiler’s ability to prevent some easy to find mistakes

noisesmith20:04:15

and also breaks some helpful tooling

noisesmith20:04:12

but AFAIK it is the same runtime expense as just using the var the normal way

uwo20:04:15

right. it smelled like a cowboy approach. I’m naming a var of a datalog ruleset from the client, but I can with a tiny bit more code just pass a keyword over instead of a symbol, and use that to lookup the var

noisesmith20:04:38

if you need a data structure to do lookups on, namespaces are silly for that

noisesmith20:04:42

use a proper hash map

tbaldridge20:04:07

To be exact, @uwo, it's about the cost of two hashmap lookups and then a normal deref

tbaldridge20:04:24

so not slow but yeah, I wouldn't do it a lot

Lambda/Sierra21:04:02

@roberto @tbaldridge I said "Don't create protocols with only one implementation." If you have a test/stub/mock version and a "real" version, that's two implementations, which is fine.

roberto21:04:31

Thank You for the clarification