This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-09-07
Channels
- # admin-announcements (2)
- # arachne (1)
- # bangalore-clj (2)
- # beginners (39)
- # boot (349)
- # cider (31)
- # clara (2)
- # cljs-dev (9)
- # cljsjs (67)
- # cljsrn (7)
- # clojure (300)
- # clojure-art (4)
- # clojure-greece (11)
- # clojure-hk (3)
- # clojure-israel (1)
- # clojure-italy (17)
- # clojure-japan (1)
- # clojure-russia (33)
- # clojure-sg (2)
- # clojure-spec (41)
- # clojure-uk (86)
- # clojurescript (123)
- # clojurex (3)
- # code-reviews (1)
- # component (6)
- # crypto (1)
- # cursive (36)
- # datomic (32)
- # devcards (3)
- # emacs (11)
- # events (3)
- # funcool (4)
- # luminus (10)
- # om (28)
- # onyx (88)
- # pedestal (2)
- # re-frame (84)
- # reagent (7)
- # ring-swagger (3)
- # specter (33)
- # sql (2)
- # vim (21)
@bwstearns not open, but there’s https://www.pdfdata.io/
@bwstearns there’s also https://www.snowtide.com by Chas Emerick (same person doing http://pdfdata.io)
if anyone was wondering more about what ghadi was talking about (seq vs reduces) hiredman has written a good piece here: https://ce2144dc-f7c9-4f54-8fb6-7321a4c318db.s3.amazonaws.com/reducers.html (which was linked from here: http://paul.stadig.name/2016/08/reducible-streams.html)
What’s the appropriate way to reference a Java enum that has a period in it, e.g. as found here - http://icu-project.org/apiref/icu4j/com/ibm/icu/util/ULocale.Category.html
Category is a nested class of ULocale
@surreal.analysis: try using a $ in place of that period
Whenever I get stuck on a class name like that, I just list the contents of the jar file (jar tf filename.jar | grep ClassIWant), or look for the .class file if it's a project I'm building. The filename of the .class file gives you the string you need
Does anyone have a really elegant solution to the requirement of “randomly intersperse items from this short list in amongst this longer list to create a single list”? I have been thinking about it in another programming language and would love to know how I could do it with Clojure with a view to proving the point that “Functional is better at this stuff”, but I am rusty and never got good at being properly Clojuric, so I am stumbling and producing code (in my head) that feels far more clunky than my OO / Procedural approach.
(it is acceptable for not all of the items in the shorter list to actually be used, but it is preferable that they are)
Depends whether you want all the elements in a random order or the shorter list interspersed into the longer list
@cddr @seancorfield - Just tried it out to be sure and it’s not cheating, but it’s not in line with the spec as it disrupts the overall order of the longer list.
Right, the latter is what you really want...
@seancorfield - Yes; the shorter list somewhat randomly interspersed into the longer list.
I'd have to think about that... interesting problem...
@seancorfield - Yeah, I thought so too...
Although I'd be more interested in a real world problem and that doesn't seem real world?
and the proof to me that I am right to feel I am not a Clojuric programmer was that I was able to run up a reasonably elegant solution in Python Ruby and PHP easily, but Clojure ideas that I had seemed clunky.
@seancorfield - It is actually a real-world problem
Our stack where I work is PHP / Python, so I gave the devs working on it an example in PHP in the end, though I brainstormed it in Ruby to being with.
What is the real world problem?
Take a list of DB results that is 20 results (page limited), and interleave a smaller list of result into the data for display purposes that are the result of a different query
The idea is to instil FOMO in the user by showing them things that they could have bought but that have already been snapped up by another customer, but do that with real sales data rather than faking it.
Here’s the example that I gave my devs to work from in PHP: https://gist.github.com/maleghast/453ec0b99212ef4e1ef1c619cd60d199
Ah, so you don't have to use all of the shorter list? Just some of them?
It felt__ as though it ought to be something Lisp, and in my world that’s only Clojure, should be good at, but I’ve never really had the time and space to use Clojure enough to become a Lisp-y thinker… So I thought I would use this as a learning experience 😉
@maleghast if I'm reading your php correctly, there's a 50% chance a random item from data_second is placed between each element of data_first?
Yeah, I can imagine a number of solutions to this problem... not sure what would be the cleanest...
if data_second is half the length of data_first, I suppose
@codonnell - Yep it’s a naive algo, just designed to show some of my devs how one might__ approach it - they were saying it could not be done...
assuming you have no nil
elements... you could randomly insert nil
into the shorter list to make something at least as long as the other list and then mapcat
them producing pairs for non-`nil` second elements or just the first element (in [ ]
).
(remove nil? (interleave coll1 (shuffle (interleave coll2 (repeat nil)))))
would give you a 50% chance of interleaving an element from coll2 in between elements of coll1, assuming your collections do not contain any nil entries.
@codonnell - I’m not claiming that it’s complete or particularly good to be honest, but I was able to run up a solution in about 10 minutes, hand-balling all the dummy data that gave a different answer form one run to the next, so it got them to see that it’s not that__ hard… 😉
@codonnell shuffle
won't work -- it loses order
I think we can change the order of coll2?
ah, ok...
might be cleaner with ->>
then, yes, that will work... you can adjust the nil
list to a given length based on a computation of ratio between the two list lengths
yeah, instead of (interleave coll2 (repeat nil))
, you could write (concat coll2 (repeat n nil))
, where you adjust n
based on the probability of not adding a coll2 element
It would be good to be able to make it a bit more “random” i.e. have list1-item, list2-item, list2-item, list1-item, list2-item, list1-item, list1-item, list1-item, list2-item, list1-item, list2-item, list2-item
(->> (repeat n nil) (interleave coll2) shuffle (interleave coll1) (remove nil?))
Given an appropriate n
that will be suitably random.
(given that order doesn't matter for coll2
)
if you want to allow adding multiple elements of coll2 in between, you could use reduce
to take a random number of elements from the shuffled and "nil-enhanced" coll2 each time you add an element of coll1
Here’s the first list (dummy / test data):
[{:status "active", :name "Tom Sawyer", :telno "+639876123456"} {:status "active", :name "Dave Sawyer", :telno "+639877123456"} {:status "active", :name "Gertrude Sawyer", :telno "+639878123456"} {:status "active", :name "Stephanie Sawyer", :telno "+639879123456"} {:status "active", :name "Dwight Sawyer", :telno "+639880123456"} {:status "active", :name "Eleanor Sawyer", :telno "+639881123456”}]
Here’s the second:
[{:status "sold", :name "Huckleberry Finn", :telno "+639890123456"} {:status "sold", :name "Jemimah Finn", :telno "+639891123456"} {:status "sold", :name "Dorothy Finn", :telno "+639892123456”}]
If you pass them in with a value of 10 for n you get:
({:status "sold", :name "Huckleberry Finn", :telno "+639890123456"} {:status "sold", :name "Huckleberry Finn", :telno "+639890123456"} {:status "sold", :name "Jemimah Finn", :telno "+639891123456"} {:status "sold", :name "Jemimah Finn", :telno "+639891123456"} {:status "sold", :name "Dorothy Finn", :telno "+639892123456”})
This is what I get:
({:status "active", :name "Tom Sawyer", :telno "+639876123456"} {:status "sold", :name "Jemimah Finn", :telno "+639891123456"} {:status "active", :name "Dave Sawyer", :telno "+639877123456"} {:status "active", :name "Gertrude Sawyer", :telno "+639878123456"} {:status "active", :name "Stephanie Sawyer", :telno "+639879123456"} {:status "sold", :name "Dorothy Finn", :telno "+639892123456"} {:status "active", :name "Dwight Sawyer", :telno "+639880123456"} {:status "sold", :name "Huckleberry Finn", :telno "+639890123456"} {:status "active", :name "Eleanor Sawyer", :telno "+639881123456"})
Here's what I got with n
equal 10
:
({:status "active", :name "Tom Sawyer", :telno "+639876123456"} {:status "active", :name "Dave Sawyer", :telno "+639877123456"} {:status "active", :name "Gertrude Sawyer", :telno "+639878123456"} {:status "sold", :name "Jemimah Finn", :telno "+639891123456"} {:status "active", :name "Stephanie Sawyer", :telno "+639879123456"} {:status "sold", :name "Huckleberry Finn", :telno "+639890123456"} {:status "active", :name "Dwight Sawyer", :telno "+639880123456"} {:status "sold", :name "Dorothy Finn", :telno "+639892123456"} {:status "active", :name "Eleanor Sawyer", :telno "+639881123456"})
So if you only got sold
entries @maleghast maybe you did something wrong?
I think you put the second list in for the first list.
Here's my REPL session
boot.user=> (def coll1 [{:status "active", :name "Tom Sawyer", :telno "+639876123456"} {:status "active", :name "Dave Sawyer", :telno "+639877123456"} {:status "active", :name "Gertrude Sawyer", :telno "+639878123456"} {:status "active", :name "Stephanie Sawyer", :telno "+639879123456"} {:status "active", :name "Dwight Sawyer", :telno "+639880123456"} {:status "active", :name "Eleanor Sawyer", :telno "+639881123456"}])
#'boot.user/coll1
boot.user=> (def coll2 [{:status "sold", :name "Huckleberry Finn", :telno "+639890123456"} {:status "sold", :name "Jemimah Finn", :telno "+639891123456"} {:status "sold", :name "Dorothy Finn", :telno "+639892123456"}])
#'boot.user/coll2
boot.user=> (def n 10)
#'boot.user/n
boot.user=> (->> (repeat n nil) (interleave coll2) shuffle (interleave coll1) (remove nil?))
({:status "active", :name "Tom Sawyer", :telno "+639876123456"} {:status "active", :name "Dave Sawyer", :telno "+639877123456"} {:status "active", :name "Gertrude Sawyer", :telno "+639878123456"} {:status "sold", :name "Jemimah Finn", :telno "+639891123456"} {:status "active", :name "Stephanie Sawyer", :telno "+639879123456"} {:status "sold", :name "Huckleberry Finn", :telno "+639890123456"} {:status "active", :name "Dwight Sawyer", :telno "+639880123456"} {:status "sold", :name "Dorothy Finn", :telno "+639892123456"} {:status "active", :name "Eleanor Sawyer", :telno "+639881123456"})
boot.user=>
Functional Rocks! 🙂
I like the easy way to pad the shorter coll - never thought of doing that, but it’s really simple once you know how 😉
If you needed to preserve order on the shorter collection, you could use reduce
and randomly conj
the element and nil
s into a sequence to be interleaved.
Yeah, actually I was working on using reduce to do it, but I felt as though it was an over-complication for the spec...
This is a really nice elegant “this is why Lisps rock” solution and it’s a good “help me to think in lists” example - I appreciate you guys helping, thanks 🙂
Can someone tell me that I’m not going insane? This seems like a bug in checking the spec of the return value
(ns fdef-ret-value.core
(:require [clojure.spec :as s]
[clojure.test :refer :all]
[clojure.spec.test :as stest]))
(defn hypotenuse
"returns hypotenuse"
[a b]
(int (Math/sqrt (+ (* a a) (* b b)))))
(s/fdef hypotenuse
:args (s/cat :a int? :b int?)
:ret double?)
(stest/instrument `hypotenuse)
(deftest specdtest
(is (hypotenuse 33 500)))
(clojure.test/run-all-tests)
The test will definitely fail if I supply a double argument, but it always passes no matter what I do with the return argument
Instrument does not check :ret or :fn -- only :args
Huh. Thanks! What’s the best way to run tests checking everything?
Instrument verifies functions are called correctly, check verifies functions are implemented correctly (with generative testing).
Hmm, what if I just want to run my tests and verify all the specs against my regular clojure.test tests?
Remember that generative testing is not like regular (unit) testing.
I use test.check all the time, but I also have a test suite that doesn’t use test.check and I still want to be able to check everything in the spec against it
I thought "clojure.spec.test/run-tests” was the answer, as described in the docs, however it doesn’t exist
Maybe that’s the long term answer
I feel like this has been covered at length in all the material written about clojure.spec
OK fair enough.
Thanks for the answer. I’m not sure I feel the same, but I will re-read it
Rich seems very clear that there are two very different types of testing here.
fwiw, the :ret and :fn thing comes up again and again, but it's been in slack and groups. I don't think anyone's written up an official rationale for it, or at least I have managed to miss it
(It's late here so I'm too tired to debate it - catch me on Pacific day time and I'll be happy to elaborate! 😀)
Sure, thanks for the reply. I didn’t realise it has been discussed to death. I’ll find it on the google groups. I don’t believe it’s discussed in the docs.
Good night 🙂
anyone know of any resources to help me understand namespaced keywords better? I dont really get the Big Deal with them. Like, why is ::my-key different from just saying :user/my-key? what can you do with a namespaced keyword that you cant do with a normal keyword?
@idiomancy as far as i understand the idea is to have a sense of ownership for keywords when working with them in foreign namespaces. think collisions in complex hashmaps, multimethods etc.
@idiomancy both of those are namespaced keywords. :: is just a naming shortcut.
namespaced keywords have context (a namespace) which allows people and organizations to play nicely together, just like namespaces for our code, etc
because while the :: refers to autoexpanding to the current ns, "namespacing" your keyword need not have anything to do with a valid ns
yes there is confusion between the namespace part of a keyword (maybe better called a “qualifier”)
that latter part is the real confusion, because im starting to get into the libraries that use "namespacing" pretty cleverly
hi.. has anybody written any blog post on adding repl endpoint in clojure server projects
? i have a clojure project and i want to access prod
instance via cider repl
from local machine..
@idiomancy That’s why in clojure.java.jdbc
I chose :qualifier
as the option for "namespacing" the keywords produced for column names /cc @alexmiller
@ashishnegi The tools.nrepl
README has a very brief example https://github.com/clojure/tools.nrepl#embedding-nrepl-starting-a-server
and if you want full CIDER middleware, this example should help https://github.com/clojure-emacs/cider-nrepl#via-embedding-nrepl-in-your-app /cc @ashishnegi
We do something very similar to the latter inside our long-running processes. Being able to cider-connect
from Emacs directly into a process in order to debug (and fix) a production problem can be very valuable. Although patching live into a production process via the REPL can be risky — "With great power comes great responsibility!" 😈
thanks @seancorfield i becomes very difficult for a newbie to understand so many nrepls.. but this is helpful..
Hey guys. I'm reading CftBaT and am at the chapter about concurrency
Aside from the memoization part, how does is a delay
different from a normal function?
The body of the delay
call won't be evaluated until forced. What you're writing inside there is basically the structure of an expression that can later be unpacked and evaluated only when needed.
You can even see what the implementation is: https://github.com/clojure/clojure/blob/clojure-1.8.0/src/clj/clojure/core.clj#L727
It wraps the body in a nullary function, a cheap way of saying 'this expression already has what it needs to run, all you have to do is kick it off when you need'.
@fellshard Although, isn't that exactly what a function is?
A bunch of expressions waiting to be run?
Which gives other functions a little more info about whether the result has been forced or not
So, essentially, the big thing is the memoization
That I can realize it from different parts of the app, and I know it'll only run the first time
The delay itself isn't too different from just waiting and then calling a function (unless you're referring to the Delay wrapper, in which case, yeah)
It's mostly the laziness - it won't be evaluated unless needed, and I can tell if it has been forced
So it's a nullary function with a bit of wrapping context... just like its implementation implies 🙂
So essentially, delay
is a wrapped, memoized function, future
is JavaScript's Promise
primitive, and promise
is similar to JavaScript's Promise.deferred
(in that you provide the value from the outside, rather than from the inside of the future
creation macro)
Am I close?
future
is more like a multi-threaded language's async-await
in some ways, since it explicitly invokes the body in another thread
And hands you back an object that will contain the result when that thread's task completes
https://github.com/clojure/clojure/blob/clojure-1.8.0/src/clj/clojure/core.clj#L6810
Yeah, it's a wrapper around an atom. With both future
and promise
, trying to realize a value before one has been generated will cause you to block and wait for that value to arrive.
promise
here isn't necessarily as robust as JS' tend to be. It's not worried about success or failure, just about the 'assign once' and blocking mechanics.
In JavaScript, you would use a Promise similarly to how you'd use a future
It gives you "hooks" to call if/when the promise is successful or rejected
But the entire logic to be run under that Promise object is contained in the Promise constructor
(in this case, the (future)
macro)
But with (promise)
it just waits and does nothing until you deliver to it from the outside (be it the main thread or another thread)
So it's essentially an orchestration tool
The concept of forcefully blocking is still foreign to me, I have to admit.
That's probably because I main JS
And in JS blocking == death.
Is there a mechanism like (promise-all p1 p2 p3) ;=> Promise([res1 res2 res3])
?
Or future-all
rather
Yeah, it does.
Welp, time to get my hands dirty
Thanks 🙂
in core.async
is it good practice/makes sense to do (async/alts! [timeout-ch [out-ch val]] :priority true)
when putting values in a channel to avoid that the put blocks forever?
I think it does, you might look at offer!
, which I think is a newish addition to core.async
@hiredman thanks I am looking, it definitely makes no sense with an async >!
or put!
but I am inside my own thread and therefore using the !!
variants
depending on what you are going to do if a put "fails" you might also just use a dropping or sliding buffer
so offer!
can be an alternative to the timeout check
yes, I see, I guess it can be used outside a go
block as well (more like my use case) but as a way to check if you can put and act if not
this seems too verbose to be right…
(if (or (seq? data) (list? data) (vector? data) (set? data))
data
(vector data)))
Shower thoughts - what if a lisp-oriented VCS was developed to run inside a Clojure application?
Might be able to adapt something like Git's graph model, with multiple users pushing and pulling from the live application's repl, or even updating the application to run a specific commit. Hmm.
> However what happens if two libraries modify the same map? On the same key? Then we are in trouble, because one overwrites the other. clojure doesn't modify maps, it produced new maps based on existing values, so how could they you face a problem of "overwriting each other"??
That doesn't make much sense to me either @idiomancy, what's the context?
I guess it's talking about namespaced keywords?
the source article: https://kotka.de/blog/2010/05/Did_you_know_III.html
Oh right. Well, maybe if you think of it like two different libraries attach different semantic information to the same keyword
if anyone can point me to another one that lays out what the idea behind them is, and why they are used like CRAZY in core.spec, I would devour it
@idiomancy it's talking about if the map were to pass through multiple functions, making it into a new map along the way. you're right that the language there is bad, because it doesn't 'modify' shit
Like maybe they both use a :user
key, but library 1 expects a user-id integer and library 2 expects a map with a session id in it
@idiomancy my favourite example is middleware. you chain a bunch of middleware functions together in an arbitrary order. they all add things to the map and pass it on to the next middleware function (or rather, they return it and something else does), namespaced keywords ensure that the different middleware functions don't interfere with each other despite working on the 'same' map
Yeah, that's a good example
@timgilbert isnt using those two maps together an error, then? isnt that something that should just be taken care of by a core.spec or something?
ring is a good example
the request and response maps are giant buckets of un-namespaced keys right now
Well, it's just data. core.spec could definitely help you sort if out, but maybe one of your libraries doesn't offer a spec
if middleware used qualified keywords those could be given some context and made a little less fragile
what i meant was that using two peices of code that have different expectations for a value seems intrinsically like an error, not a namespace collision
(One of the good design decisions in spec is that it forces you to namespace your keys anyways)
@idiomancy: Good interview with Rich if you want to get deeper into some of the rationale behind it http://blog.cognitect.com/cognicast/103
The problem is more interoperability between two libraries that don't necessarily know anything about one another, but happen to have a collision in a key they use
it seems kind of weird that a chain of middleware functions would all just add keys to the same map.. why not just have a different map for each middleware?
wouldnt the latter even give you the benefit of retrospection on the order of operations for free?
using qualified keywords means you can attach semantics to keywords with context
It's not the only way to compose a bunch of middleware, but it's a common one, cf https://github.com/boot-clj/boot/wiki/Tasks
few answers: 1. because that's just the way it is right now 2. flat maps are 'simpler' than nested maps. to read from, to add to, to refactor usage of 3. because you would have the same issue with the keyword that references the middleware's middleware-specific map 😛
Pedestal takes a somewhat different approach with interceptors, which re-frame has now also adopted. But anyways the problem is the same, two things using a single key with different semantics
well a namespace for a keyword is a pretty simple idea. I'd agree that it's definitely not succinct though. but the latest version of clojure has been adding syntax to help with that
hmm.. yeah, I particularly cant see the "ease of access in a flat map" argument. If you just joined all of your maps together, and nested the data based on namespace, then what is the difference between, say
(get map :ns/value)
and (get-in map [:ns :value])
?three backticks, not 2
@idiomancy ok, now try dissoc :value
in :ns
if I wanted to do something with every value in the map, or if I wanted to get the values for the keys [:basic/user :ssl/token :csrf/token]
@tanzoniteblack dissoc-in?
doesn’t exist
not sure the history of why it doesn’t though
and bfabry’s point is much better
@bfabry I suppose it makes sense, but that seems like it could just as easily have been solved by making a select-keys variant
also what if I want to change the key name in the basic middleware from ::user to ::user-name
like, I could selectively find and rename all the :basic/user's to :basic/user-name without ever worrying that I'd break :ssl/user
ohhh, I see. but within a codebase, you dont have that problem, I thought. its all just named ::user
I might have that problem if my codebase consumes both the basic and the ssl middleware
hahaha, youre describing a clojure codebase larger than I have ever worked with, so its very possible I'm just out of my depth here
well, for something like :id i'd use ":entity-id" in general, if im expecting a completely different concept of the value
I mean it's definitely true that you can get largely the same benefits through using longer more descriptive keywords. but no one does. clojure is just adding a bunch of tooling and encouragement to make it easier and more common
If I have a list of: ‘(+ 2 3) then I can eval it and it will return 5. What if I have a list of ‘(+ 2 x) ? Is there some way I can eval it in an environment where I can bind the symbol ‘x to a value? The best I’ve managed is:
(def x) (with-redefs [x 3] (eval the-list))
but this is hard coded to a symbol called “x”, and it pollutes the environment with a var. Similarly for dynamic vars and bindings.
I also can’t use a macro, because the-list will only get the expression at runtime, and the symbol isn’t fixed either.
I feel like I’m missing something obvious here….@quoll where are you getting the list from?
this is fundamentally a runtime thing?
(eval `(let [~'x ~3] ~the-list))
I’ve been tempted to do this, but the expression has the potential to be more complex. It will also be inside a tight loop
@quoll what does tight loop mean? You'll be evaluating the same expression multiple times with different values for x
?
oh good
(let [f (eval `(fn [~'x] ~the-list))] (f 3) (f 4) (f 5) etc...)
what you want to do is walk the expression finding free variables, then create a function using eval, that has the free varaiables as parameters
like what @gfredericks said, but not hard coded to x
@quoll is the query language just arbitrary lisp? or is it a really short whitelist of ops?
over time, more ops would be implemented. No, it won’t allow someone to create arbitrary lisp
I guess eval might be fast enough for an http request
it's less than a millisecond
little languages like that ofter aren't difficult to compile by hand though
often*
eval is calling in to the clojure compiler, compiling to jvm bytecode, loading the class containing the bytecode, and running it
I was also trying to avoid evaluating the expressions by hand. I’ve done that before in Java, but since I had this magical thing called “eval” I was hoping to leverage that
Hmmm. It might be faster to compile a class (slow, but only once), and then call an invoke function on it (fast)
@quoll that's essentially what @gfredericks last snippet did
in general, I would be suspect of any approach that translates your queries in to clojure, then calls eval
you could use tools.analyzer for the free variable analysis, but in that case you will definitely want to compile and re-use, tools.analyzer is complicated
if you have a simpler language to start with, you can do that free analysis first, before translating to clojure
If the language doesn't have bound variables at all then stupid approaches can work