This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-03-07
Channels
- # aleph (4)
- # arachne (2)
- # aws (2)
- # beginners (42)
- # boot (4)
- # cider (47)
- # cljs-dev (352)
- # clojure (278)
- # clojure-dusseldorf (6)
- # clojure-italy (6)
- # clojure-russia (1)
- # clojure-spec (15)
- # clojure-uk (94)
- # clojurescript (197)
- # community-development (34)
- # cursive (3)
- # data-science (1)
- # datomic (64)
- # emacs (3)
- # figwheel (16)
- # fulcro (7)
- # hoplon (5)
- # jobs (3)
- # luminus (3)
- # mount (2)
- # nyc (1)
- # off-topic (31)
- # onyx (22)
- # parinfer (1)
- # protorepl (7)
- # re-frame (9)
- # reagent (61)
- # ring-swagger (3)
- # shadow-cljs (149)
- # spacemacs (18)
- # specter (4)
- # timbre (1)
- # unrepl (38)
- # vim (17)
- # yada (14)
it's an easy function to write, but not built in, I don't even know if it has a standard name
it is almost what cond-> does at each step though
FWIW, at World Singles, in our worldsingles.clojure.extensions
namespace, we have condp->
and condp->>
for exactly this purpose (where the p
signifies the expression threads thru the predicate as well).
in mimicing pointless code, one can define:
(defn ite [pred t e]
(if (pred x) (t x) (e x)))
then the above becomes (ite pred f id)
you also need to pass in x
and define id
though theoretically it's better to have thousands of functions that operate on 'x' 😉
I mean it's funny that it's missing
because other similar cases are covered
yeah, id
is aliased as identity
, and the above returns a function, which yo pass x
to ... or you can pass this function to other function combinators
clojure seems to not like styles of code where functions ar ecombined without explicity naming their args
the only reason I nitpick this non-functioning code is that once you mutatis mutandis turn it into something that works, it's as verbose as if and harder to read
I think it's better in situations like:
(map (ite pred f id) ...)
vs
(map #(if (pred %) (f %) %) ...)
this actually reminds me of why I started writing code like this -- clojure doesn't let us nest #( ... % ...)'s 🙂
the downside is that I can understand the bottom one in 1 second, the top one I have to go look up
I actually wish there is a standard library of function combinators
for combining small functions into larger functions
wait, is "ite" some standard thing? google isn't helping
'ite' is an acronym that has to be (as you say) memorized but it also misses the secondary meaning that a word like 'map' infers upon the funtcion
@noisesmith: no, it's not, it's a short hand for If Then Else
you've got a lot of memory, just type 'if-then-else'. At least that's what I'd do. Names are important (per Tellman 🙂 )
OK - that doesn't scale, I can't memorize a new acronym for every clojure user, so for unfamiliar things as schmee says I need to look them up
@the2bears before reading SICP, map
to me was a piece of paper with roads + mountains + rivers drawn on it;
Haskell has it's list of function combinators
J has it's adverbs/dyads ...
I've been punished by the experience of trying to work on code bases full of idiosyncratic abstractions with weird names, so I get triggered by this stuff
@noisesmith: I've been trying to rite clojure as concise as APL. good thing we aren't coworkers 🙂
Yes @qqq, but as I said there's a meaning to the word, that once understood infers meaning on the function. Once learned you're very unlikely to forget. Not the same with 'ite'.
@qqq that's terrible. up to a point redundancy acts as error correction for human readers. the primary audience for code is humans
I've actually worked on codebases like this and it's not fun at all
I would have difficulty remembering functions that I actually wrote, with acronyms like this 🙂
@the2bears: I believe the APL approach is: most functions are 1 line, most programs are 1 page, so just rewrite it
@the2bears unless you poop out a new php app every day and never look at any of them again haha
FWIW, at World Singles, in our worldsingles.clojure.extensions
namespace, we have condp->
and condp->>
for exactly this purpose (where the p
signifies the expression threads thru the predicate as well).
We keep that namespace very slim.
noob question: from what I can tell, clojure has logic functions and
, or
, not
but there's no xor
. I see that there's a bit-xor
but that's not what I'm after. I know that xor
is logically just (defn xor [p q] (or (and (not p) q) (and p (not q))))
but I was wondering if there's any deeper insight into why maybe it's not in the core libraries?
The only other option I can think of is to take the args, cast them into booleans, then ints, then call bit-xor. That’s just about as ugly though
That being said, I’ve never needed logical xor in my 8 years of Clojure work, so it’s probably not common enough to make it into core.
yeah bit manipulation is definitely a more common use case for it
Actually I don’t think I’ve ever used it in any language. Just bit-xor
actually that's a really good point... lol
(contains? #{[true false] [false true]} [(boolean x) (boolean y)]) - likely not the most efficient, but it's readable
oh, that's better (not= (boolean x) (boolean y))
oh nice. don't think I've seen not=
before
@chrisbroome And just so you're aware, Clojure's and
and or
don't necessarily produce Boolean values -- they produce truthy / falsey values.
yeah that makes sense
actually that's a lot like JavaScript's &&
and ||
(I do javascript in my dayjob)
Is it essentially a type check? You assert this or you assert that but not both? A makeshift tagged Union?
Yeah it’s really an untagged union where the unioned types happen to share some structure.
Yeah our old job we were doing that in c# and it was super nice to add f# to the codebase
Hi everyone, I would like to ask for your opinion on interfacing in Clojure. Previously, when working on an OOP language such as Java, my first instinct when dealing with "layered" code is by using interface. Each layer is communicating by interface, and which implementation is used usually determined by which concrete class implementing those said interface is injected. So that my code would pretty much like:
class OuterLayer {
private InnerInterface innerLayer;
public OuterLayer(InnerInterface injectedObject) {
this.innerLayer = injectedObject;
}
public int calculateSomething() {
return this.innerLayer.Calculate();
}
}
How does Clojure deals with this kind of needs? I have a logic layer that really doesn't care which kind of storage that the storage layer use as long as it returns the data that it needs. I've managed this far with Clojure (which is not that long) by ignoring the interfacing concept, I just require a namespace that provides functions for storing objects and call its functions. But lately I've been thinking that what if I decided to discard those namespaces, would simply requiring different namespace which has exactly the same function names and signatures suffice?
Isn't that what Protocol does? How would I "inject" and decide which implementation to use? Is Dependency Injection not idiomatic in Clojure?@hawari.rahman17 yes protocols can be used in the exact same way if you need that. Although I would recommend only using this kind of abstraction if you are already sure you will want to swap implementations a lot. Otherwise just do one implementation for now and refactor it in the future
I won't probably change the implementation that often, it's just probably a habit of making my pieces code as independent as possible and using "contracts" as intermediary that drives this.
@hawari.rahman17 DI in clojure consists of writing your consumer to use protocol functions, and then passing in your implementation as an argument, this is how all the built in clojure collections work (except interface instead of protocol, which is a minor implementation difference)
but, most of the time, we can write the most flexible and correct code by using the clojure.core functions that are all built on top of the collections (conj, get, assoc, map, filter, reduce etc. etc.)
and yes, a database connection, or a file resource, etc. something that can't be modeled by immutable data in memory, is typically used via a protocol
(or in the case of our native datomic for example, used via a protocol and also modeled by immutable data in memory)
@noisesmith Yes, my primary case is usually related to the external resource access like the one you mentioned (DB Connection, API call, etc). I have another question, though this probably fall into preference and not a concept per se.
In OOP based language the injected object is usually stored to a class field. How should I do it in Clojure? Should I use def
in the consumer namespace?
namespaces are not classes, don't use them that way
def is for creating globals, avoid putting anything mutable in a def
the idiomatic thing is to provide the implementation as an argument to the code that uses it
there are libraries like stuartsierra/component or integrant for the pattern of initializing stateful resources as you start a program, and passing them to their consumers
for resources that represent something fallible that would be reinitialized while running (db connection a good example of that) you can use a mutable container holding the resource, and have code that re-establishes it on failure
but that mutable container still shouldn't be a global
At this point in our development, I don't think a major refactor by using component
or passing the implementation as an argument is desirable. What about dynamic vars in the consumer namespace? Problem is I don't have the idea on how to rebind it, or exactly where would I determine which implementation this dynamic vars will be bound to.
this is why passing it as an argument simplifies things - "somebody" passes the implementation to some function, which in turn uses clojure.core/binding for a dynamic var (if that's how you want to do it) and calls the rest of the code inside that binding blocks
passing an implementation as an argument is the simplest and most general thing - all other options (globals, binding blocks, mutable values etc.) can trivially be implemented based on an arg passed in
the others are not as flexible
Retrofitting a large app with component can be a long/painful process for sure (had to do it). But it's worth it imho
I've never had a chance to look into component
, but I'm curious though, as it seems that a lot of people praise it. Would you care to tell the story of your usecase with component
?
@hawari.rahman17 sometimes putting the db in a mutable global actually makes sense. When you do that put it inside an atom (def db (atom nil))
and then use reset!
to establish the value (based, of course, on a value someone else passes to you)
@mpenet: not to mention, the longer you wait to use component, the larger the app you end up retrofitting (or even worse you end up with all the problems caused by hard-coded global resources)
@hawari.rahman17 if you do opt for the mutable global connection atom, at least decouple the code that needs testing from the hard coded location of that value - at the very least the core testable logic should be decoupled from globals (even if your app can handle a global)
@noisesmith The "somebody" that will pass the implementation ought to start somewhere right? Let's say my outermost layer is a ring handler, then should I bind the implementation there and pass it on to the logic layer?
right - a typical pattern is to have a function that initializes the db, then passes that value to another function that starts your http server, and passes the db as a resource to each handler as it runs
the ring spec allows (assoc request :db db) - it's a hash map, you can attach anything you need to it and it just works
it's trivial to write a middleware that takes a db object and a request handler, and returns a new request handler that passes a db to each endpoint
agreed with @noisesmith - I’ve tried a few different approaches, but in the end passing db and similar “handles” in the request via middleware is straightforward to write and easy to test
(fn attach-db [db handler] (fn handle-request [request] (handler (assoc request :db db))))
- called as (def handler (attach-db db raw-handler))
that returns a new handler that you pass to the http server
using atoms or other mutable globals is just asking for trouble
yeah - building a set of middlewares so that each request sees resources it needs works great - I'd upgrade from that to stuartsierra/component if/when you need to do things that use resources outside an incoming request (scheduled tasks, long running things that outlive requests, etc.)
Wow... yeah... I've never think about ring that way, always saw it just as an abstraction to HTTP.
Thank you @noisesmith, it has been a very thoughtful discussion with you
@hawari.rahman17 np - just remember in general that the reason clojure apis are built around functions and hash-maps is that they are both very flexible - ring didn't have to implement an extensible dep-injection api, hash-maps and first class functions do all that work for it for free
the hard part, in the end, is using that immense power thoughtfully and not making messes of untyped functions on hash-maps with data that comes from god-knows-where with who-knows-what in them haha
hey everyone, I am looking for a recommendation for a Kafka client. A quick search tells me that https://github.com/pyr/kinsky is the right way to go. It seems the most widely adopted and the most maintained. Is that right? would you rather go with interop instead?
I used interop, but used it before from java. It’s very easy this way to adopt to how you want. For example I have a way of setting the key depending on the value for the producer, and for the consumer I pass a consume function. Also I set some sensible defaults and set the correct de(serializers). This stuff is all very project-dependent and it’s nice to have it in one location, also by using interop you can easily change the kafka version you use, which is very nice since clients should not use newer versions then the brokers/servers.
+1 for interop with the official client: easy to use (can make your own helpers for commonly used things), no extra dependencies, guaranteed to be up to date with the latest features etc etc
do you mean a library like jq for Clojure data structures, or a jq-like library for JSON written in Clojure?
@grav@stathissideris was working on something like that
@grav check specter
@grav the work @dominicm is referring to was a command line tool that would integrate specter and other functionalities, and would operate a bit like jq
I started looking into it, but I didn’t get very far yet
@stathissideris Ok, well in that scenario, jq works quite well for me. But I'm looking for something that can operate on a non-string representation of the data.
@grav yeah, for me the motivation was to not have to deal with jq’s syntax and do it in the familar clj syntax instead
It is quite complex, I'll give you that. But for my usage, I only deal with a tiny subset of what's possible. Still, even that tiny subset is extremely expressive. I'd have to work hard to implement it in Clojure myself.
I’m wondering if someone is using a BPMN Workflow Engine (e.g. Camunda BPM) from Clojure?
came down to creating instances of a workflow, and reading the state of N instances later on
But the engine was deployed as a standalone engine? Or was it embedded and managed from within your clojure application?
https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.10.1.9.faload <-- does this mean JVM has a hard limit where float-arrays can not have more than 2^31 elements ?
https://stackoverflow.com/questions/3038392/do-java-arrays-have-a-maximum-size <-- agrees with you, @bronsa
@benzap: no, merely writing a mini DSL that targets the JVM, and finding that all array load ops take int
not long
there’s other limits like that that you might not expect, like the number of arguments being limited to 254
@benzap there’s no unsigned anything in java, ofc you can emulate them but it’s a bit ugly
@bronsa: ah, that's fine, since iload/fload/aload can only use byte for index, and 'this' object takes first location
every VM has that stuff though. Even Python, although Python has some weird extensions where you can prefix a number with 0xFFFF that says the next "int" is twice the size
IIRC some of the mmap stuff uses longs though, because with mmap you can map way more memory than 2^32, makes working with mmap stuff a bit of a pain.
it's a bit weird since I have now have to use 'array of arrays' since: 1. 2^31 * (4 bytes / float) is only 8 GB 2. server has > 8 GB memory 3. machine learning dataset (read only) is > 8 GB, but fits in memolry 4. one simple way to represent tensors is: a flat array + dimension + offsets for each dimension 5. ==> that now needs an extra layer of indirection, either splitting the dataset, or each 'input sample' as a float-array, then have an array of input samples
You want that anyways. Trying to store 8GB in one array is going to trash the JVM GC
Store that off-heap or something
or like I mentioned, use mmap
@benzap: the 'correct' solution probably involves cuda or jni+atlas, but at the moment, I'm curious about how far the JVM can be pushed
From what i've read, luajit is supposed to be pretty impressive in terms of VM performance
luajit/torch has a very strong following; however, I really want ML and 'rest of app' to be on the same language ... and I've already picked Clojure for 'rest of app'
I've run into a strange bug. How far reaching is using the binding
function? If I go (binding [*x* 2] & body)
and I have a deeply nested function which refers to x, will it always pick up the new value 2?
it reaches as far as possible, but may not apply if you change threads (or another binding of course)
Any help importing this java project into leiningen?
https://bintray.com/javaphoenixchannels/java-phoenix-channels/JavaPhoenixChannels/0.2.0
Tried adding [com.github.eoinsha/JavaPhoenixChannels "0.2.0"]
to :dependencies' and
{“javaphoenixchannels” “https://dl.bintray.com/javaphoenixchannels/java-phoenix-channels”}` to :repositories
@pablore By default Lein/Boot only look at Clojars and Maven Central. You'd need to add a repository definition to pick up dependencies from elsewhere.
Ah, just noticed you tried that...
@seancorfield yep, I added a link to the :repositories, but still it cannot find it
Use version 0.2
not 0.2.0
The project page does say <version>0.2</version>
-- not 0.2.0
The GitHub tag doesn't match the artifact version. That's kind of silly of them.
There's no equivalence in versions -- they're very literal 🙂
So it turns out it is dropping my binding, but I the test appears to be single-threaded
since I don't have the db set up correctly in CircleCI (should just fix that really), I just added a dirty trick to avoid running them in Circle entirely https://github.com/AndreaCrotti/just-married/blob/02d271ea5112404914e4f38812cabb28907be0a8/test/clj/just_married/db_setup.clj
checking an env variable, and composing test fixtures
however that doesn't really work, I don't think I can skip tests in this way by simply not evaluating test-fn
in one of the fixtures
any other way to do that?
@benzap another thing to look out for is lazy values realized outside the binding block
for line 6 I would check the results of those two functions, I wouldn't be surprised if hash-password returns a string and your database call returns a byte array or something like that
put a prn under the is on line 6 printing out the values of both of those functions, run the tests, see what it prints
the tests for the password hash without hashing works fine, so the db is fine, the hasher just seems to salt differently :S
That is not an appropriate link for this channel. I've deleted it (as an admin).
it seems pretty clear, what you are storing in the database is not the output of (hash-password "test")
except I can login to my website just fine with generated users, so it's the main reason I thought it was a binding issue
I just making sure it wasn't using the orginal :database-secret, so it must be rebinding it
That is not an appropriate link for this channel. I've deleted it (as an admin).
Why pprint
throws (`IllegalArgumentException No matching field found: param for class clojure.lang.Symbol clojure.lang.Reflector.getInstanceField (Reflector.java:271)`) - in third line but not second:
(deftype TestType [param] Object (equals [_ other] (= param (.param other))))
(pprint (hash-set (->TestType "a") (->TestType "b")))
(pprint (list (->TestType "a") (->TestType "b")))
Do you have a longer stack trace? (`*e` in the repl) Something must be calling =
on your TestType obj, but I have no clue why.
:cause "No matching field found: param for class clojure.lang.Symbol"
:via
[{:type java.lang.IllegalArgumentException
:message "No matching field found: param for class clojure.lang.Symbol"
:at [clojure.lang.Reflector getInstanceField "Reflector.java" 271]}]
:trace
[[clojure.lang.Reflector getInstanceField "Reflector.java" 271]
[clojure.lang.Reflector invokeNoArgInstanceMember "Reflector.java" 315]
[user.TestType equals "form-init708560037594969920.clj" 1]
[clojure.lang.Util$2 equiv "Util.java" 50]
[clojure.lang.PersistentArrayMap indexOfObject "PersistentArrayMap.java" 253]
[clojure.lang.PersistentArrayMap indexOf "PersistentArrayMap.java" 270]
[clojure.lang.PersistentArrayMap valAt "PersistentArrayMap.java" 235]
[clojure.lang.PersistentArrayMap valAt "PersistentArrayMap.java" 242]
[clojure.lang.APersistentMap invoke "APersistentMap.java" 286]
[clojure.pprint$pprint_reader_macro invokeStatic "dispatch.clj" 50]
[clojure.pprint$pprint_list invokeStatic "dispatch.clj" 77]
[clojure.pprint$pprint_list invoke "dispatch.clj" 76]
[clojure.lang.MultiFn invoke "MultiFn.java" 229]
[clojure.pprint$write_out invokeStatic "pprint_base.clj" 194]
[clojure.pprint$pprint$fn__8473 invoke "pprint_base.clj" 249]
[clojure.pprint$pprint invokeStatic "pprint_base.clj" 248]
[clojure.pprint$pprint invoke "pprint_base.clj" 241]
[clojure.pprint$pprint invokeStatic "pprint_base.clj" 245]
[clojure.pprint$pprint invoke "pprint_base.clj" 241]
I suspect that pprint
is trying to "inspect" the structure in an effort of making an output pretty
Seems plausible
(def a (->TestType "a"))
=> #'user/a
(pprint (list a))
IllegalArgumentException No matching field found: param for class clojure.lang.Symbol clojure.lang.Reflector.getInstanceField (Reflector.java:271)
But it only does it for list, not for map
Ok, the code that’s throwing the exception is checking for if (f != null) ...
(in Java)
How it got there, I don’t know, but that’s at least the immediate cause.
I probably should have inspected a stack trace before posting here but now when I read sources, it seems that there is a special case for "pprint-reader-macro" (which is a list as well) https://github.com/clojure/clojure/blob/clojure-1.8.0/src/clj/clojure/pprint/dispatch.clj#L50 (I am on 1.8)
so the "low level" cause is: ({'quote "'", 'clojure.core/deref "@", 'var "#'", 'clojure.core/unquote "~"} (->TestType "a"))
where the map is https://github.com/clojure/clojure/blob/clojure-1.8.0/src/clj/clojure/pprint/dispatch.clj#L45
@hiredman figured out the issue with the hashing. It had to do with how I was checking for correctness in the returned db value
no, it was how I was checking the hashes, I guess bcrypt has a special way that it checks
On the bright side, I think my test coverage has been getting pretty good now just trying to figure out this problem haha
hey all, this might be a dumb question -- does anyone know how i would go about doing something similar to update-in
, but one where the keyseq didn't require indexes for vectors/sequences and just updated all the values in the vector
e.g.: I have this map: (def my-map {:vector [{:field1 1 :field2 2} {:field1 11 :field2 22}]})
and I wanted to increment :field1
of both the maps in the :vector
vector. What I want to do is something like (new-update-in my-map [:vector :field1] inc)
instead of having to call update-in
twice with [:vector 0 :field1]
and [:vector 1 :field1]
i guess i can just write the fn myself, but wondering if anything else already does something similar
but they're more complicated and the key sequence is generated dynamically for a nested structure that could have vectors at any level
(defn update-in-all-vec
"same as update-in but doesn't require indexes when it comes accross a vector
it just applies the 'update-in' accross every item in the vector"
[m [k & ks] f]
(let [val (get m k)]
(if (sequential? val) ;; if val is a sequence but not a map
(if ks
(assoc m k (map #(update-in % ks f) val))
(assoc m k (map f val)))
(if ks
(assoc m k (update-in (get m k) ks f))
(assoc m k (f (get m k)))))))
this is a perfect use-case for Specter:
user=> (def my-map {:vector [{:field1 1 :field2 2} {:field1 11 :field2 22}]})
#'user/my-map
user=> (transform [:vector ALL :field1] inc my-map)
{:vector [{:field1 2, :field2 2} {:field1 12, :field2 22}]}
here’s one way to do that 🙂
user=> (def my-vector {:vector [{:field1 1 :field2 2} {:field1 11 :field2 22}]})
#_=> (def my-map {:vector {:field1 1 :field2 2}})
#_=> (defn selector [k] (path :vector (if-path map? k [ALL k])))
#_=> (transform (selector :field1) inc my-map)
#_=> (transform (selector :field1) inc my-vector)
#_=>
#'user/my-vector
#'user/my-map
#'user/selector
{:vector {:field1 2, :field2 2}}
{:vector [{:field1 2, :field2 2} {:field1 12, :field2 22}]}
feel free to ask me if you have any questions! also, the creator of the lib hangs around in #specter and answers questions there 🙂
@benzap it’s time well spent! just learning basic navigation with keywords and ALL
like I did above can solve so many problems 🙂
yes, +1 to that. i have no idea how specter actually works but i have a few paradigms i’ve figured out and i use it all over the place. Also: although it is weird and complicated, it is the rare library that is far easier to read than write, so you don’t have the problem of write-only code
agreed, that’s what I love about specter: easy things are easy, hard things are possible
that's nathan marz's thing, right
i tried to wrap my brain around it, didn't succeed
I just wish it had a more data-driven interface, that's one of the reasons I haven't learned the library yet. Function composition is just hard to debug and too opaque.
That and it lacks a join construct
maybe checkout https://github.com/tonsky/datascript
I’m following this blog post to use http://ring.util.io/piped-input-stream in concert with clojure.data.xml to emit
to an input stream as an HTTP response body. wondering if anyone knows how to compute the Content-Length header in this context without incurring the cost of calling (-> some-parsed-xml emit-str .getBytes count)