Fork me on GitHub
Chris O’Donnell02:07:39

@josh.freckleton: I guess keywords can have spaces in them

Chris O’Donnell02:07:13

=> (map (fn [[k _]] (name k)) mapy)
("a" "a " "a  ")


A lot of libraries rely on being able to call keyword on arbitrary strings...


There’s a lot of keywords you can construct programmatically that are not legal in the reader.


(keyword "") => : for example


so is this a bug or a feature? Seems like it ought to be made consistent one way or another


It’s a feature. It’s been discussed many times over the years.


At one point it was discovered that the reader accepts a lot more keywords than it is documented to accept and there was a change to tighten that up — but it broke a LOT of libraries, including some very popular high-profile ones, that assumed you could write literal keywords with numbers etc (`:1`, :>) so that change was reverted.


And Clojure/core have said several times that it’s intentional that keyword can be used to construct (pretty much) arbitrary keywords, that you can then call namespace and name on to turn back into strings.


Consider :a/b = (keyword "a" "b") = (keyword "a/b") -> (juxt namespace name) => ["a" "b"]


(similarly for the symbol function)


And that’s enshrined in stuff like test.check and clojure.spec too:

boot.user=> (require '[clojure.spec :as s])
boot.user=> (s/def ::a keyword?)
boot.user=> (s/exercise ::a)
([:_ :_] [:!/Z :!/Z] [:J :J] [:k?q+/h+cX :k?q+/h+cX] [:-*w7 :-*w7] [:C- :C-] [:HO+1.E9n4vc.+?cy-x/wy :HO+1.E9n4vc.+?cy-x/wy] [:?y0?j!v.+cOru?6.B*.w9+.Vp?P8.++*VT*0-.i/pq :?y0?j!v.+cOru?6.B*.w9+.Vp?P8.++*VT*0-.i/pq] [:?k.K*.!dKX!9/-U0E0!s_9 :?k.K*.!dKX!9/-U0E0!s_9] [:-- :--])


thanks for the explanation


or with a conformer showing the namespace/name decomposition:

boot.user=> (s/def ::a (s/and keyword? (s/conformer (juxt namespace name))))
boot.user=> (s/exercise ::a)
([:* [nil "*"]] [:d [nil "d"]] [:u? [nil "u?"]] [:Dr0.-/V? ["Dr0.-" "V?"]] [:_b_.?!++0/L6 ["_b_.?!++0" "L6"]] [:V+O!c_.gY5D.G6o.v!Av/R ["V+O!c_.gY5D.G6o.v!Av" "R"]] [:W-p?D.!/Ou+O7s ["W-p?D.!" "Ou+O7s"]] [:W*9.D1F2.s?531/fm ["W*9.D1F2.s?531" "fm"]] [:!?f2 [nil "!?f2"]] [:y.ylfs!ow.T_?R*RL-._e??4vQ/fjc4! ["y.ylfs!ow.T_?R*RL-._e??4vQ" "fjc4!"]])

Chris O’Donnell03:07:33

@seancorfield: interestingly, the clojure.spec generator for keyword? doesn't seem to generate keys with spaces in them


@codonnell: yeah, it only uses these special characters (elements [\* \+ \! \- \_ \?])


(took me a while to find that deep down in clojure.test.check.generators)

Chris O’Donnell03:07:47

that seems arbitrary

Chris O’Donnell03:07:54

nice find, by the way


I’m trying to write a stateful transducer that builds up a list of stuff and then during the flush pushes that collection through the downstream transducers. Something like this:

(defn delay-events-xf [xf]
  (let [events (volatile! [])]
      ([] (xf))
       (loop [event events]
         (xf result (first event))
         (recur (rest events))))
      ([result {:keys [delay] :as event}]
        (if delay
          (do (vreset! events (conj @events event))
          (xf result event))))))
But obviously the result should change during the flush step, so doing that loop is probably wrong? I have a suspicion this is just totally out of scope of what transducers are supposed to do...


It’s from the definition of symbols here > Symbols begin with a non-numeric character and can contain alphanumeric characters and *, +, !, -, _, ', and ? (other characters may be allowed eventually).

Chris O’Donnell03:07:38

That makes sense. So the same generator is used for symbol? as well, probably.


@lfn3: If each event is a map, shouldn’t events be an empty vector to start with?


Also, your loop has no recur so it just runs the first delayed event through the xf?


Yeah, sorry I hacked apart some ‘working’ code to produce that.


Edited the code above.


Maybe (reduce xf result events) is what you’re looking for?


Yeah that makes sense. I’ll give it a go.


So for the record, that worked, but you can also just keep calling xf:

(defn delay-events-xf [xf]
  (let [events (volatile! [])]
      ([] (xf))
       (when (seq @events)
         (xf result (first @events))
         (vreset! events (rest @events))))
      ([result {:keys [delay] :as event}]
        (if delay
          (do (vreset! events (conj @events event))
          (xf result event))))))




@seancorfield: @codonnell Aw sorry I missed out on the keywords discussion, your answers were both enlightening and unintuitive, so, insightful! 🙂


hi! I have a newbie question


I’ve just read a form like `


(let [cancel-ch (a/chan)
        times-fn (^:once fn* [] times)] … )


what do ^:once mean? and what’s fn*?


I still don’t understand why not

(let [times-fn (fn [] times)] …)


There's a long history of Lisps accepting a lot in Symbols. You can do some truly perverse things in CL (which gets even more confusing because of the reader- I thought CL was case-insensitive for longer than I'd like to admit.)


There are some legitimate reasons to be permissive in this regard. Whether they are good enough to justify CL's level of permissiveness is another question.


@esirola If you are a beginner you might not want to worry too much about this. And I'm very much a beginner myself, so take this with a grain of salt, but...


@esirola: Doesn't sound newbie to me! I've certainly not encountered code using fn*. It's possibly to do with performance. The :once metadata is an advanced feature to do with garbage collection. I haven't read this too deeply, but if you'd like to do deeper, cgrand covers it here


fn* is, if I understand correctly, used in bootstrapping the clojure compiler.


That is to say that clojure is largely written in Clojure, but it has to start somewhere.


And the once metadata is as domincm says.


So if I had to guess I'd guess that the person who wrote the piece of code you are looking at really cared about erformance in that bit of code.


Oh- OK, that makes some sense, actually.


definitely not for a newbie, but it’s good to know there’s a potential leak in async/onto-chan, I’m going to use channels for data management pipelines


OK- also, sorry- thought I was in the beginners channel.... where I thought this was an odd conversation.


@esirola, @amashi - yep, was definitely to do with the memory leak rather than performance - that code's used for potentially infinite sequences so needs to not hang on to the whole sequence of times


IIUC (@bronsa?) it's because when (fn [] ...) closes over values, it can't garbage collect them until the function itself is garbage collected


@dominicm: thanks for the pointer, I’ll read it


yep, correct


so that work around closes over the function rather than the value


whereas supplying the :once metadata means that it's set to null after it's first used, meaning we don't hold on to the head of the lazy seq, so it can be GC'd as it's consumed


so yes, not a newbie question at all 🙂


Yeah- I was talking in that channel and failed to realize I had changed channels.


@jarohen, @bronsa oh ok now I think I see the point, thanks a lot for the help!


folks, possibly a newbie question to do with types. (warning, evil stuff ahead). I've been playing with net.cgrand.spreadmap, mainly because it's fun, but there is a pretty big problem I've found - it doesn't follow cell references to different sheets inside the same workbook. The code hasn't been updated in ages so it's probably just an api change, but I'm trying to work out how to fix it. The error message doesn't give me a decent stack:

java.lang.AbstractMethodError: net.cgrand.spreadmap$workbook$reify__1921.getExternalSheet(Ljava/lang/String;Ljava/lang/String;I)Lorg/apache/poi/ss/formula/EvaluationWorkbook$ExternalSheet;
which is pretty useless. I would -guess- it's failing looking up the sheet name in java but the funny thing is, I can't for the love of me find out how to set breakpoints where it does this. Is there a way to get the full stack trace for this? Like 'last exception' somewhere?


@adamw: try (.printStackTrace *e)


or indeed, just *e 🙂


beautiful, thank you!!


I am using the salesforce RESTful API and I am trying to download 130,000 records. Problem is the API limits each request to 500 records so obviously this is taking a LONG time to download. Does anyone know a work around for this with salesforce?


Another really obvious quesiton I'm sure - where are all these G__33 variables coming from? Is this some kind of special reify thing?


prolly (gensym), meaning the code was macro generated or something


@mpenet: oic, so that means that basically that file is a clojure type presenting all the methods exposed in the Java class 'EvaluationWorkbook'?


that explains a lot, the error is otherSheetIndex = targetEvaluator.getSheetIndex(externalSheet.getSheetName()); but there is no getSheetIndex on the reify EvaluationWorkbook in the clojure source


kinda yes, that function returns an anonymous class instance that implements it


thank you!


oh no I lie


these two exist


perhaps the type is different. Hmm, back to the debugger


oh of course. It's not an EvaluationSheet it's an EvaluationWorkbook.ExternalSheet, right, got it.


nope that;s not it. Never mind


Experimenting with clojure.spec and reading Datomic docs. Is spec'ing :artist/name some kind of bad practice? Might seem like a strange question, but all examples I've seen use namespace-qualified keywords like ::name, but I feel like I would rather start out with specs for the problem domain rather than mixing in namespaces of my implementation domain everywhere. Any advice appreciated.


:artist/name is namespace-qualified.


Well, if spec ever become popular we ll have collisions between kw. Ex an auth framework using :auth/ or :user/ as a dependency... Prolly better to use project specific ns (same as lein project ids).


That s one of the reasons I dislike the whole registry concept in spec. But it seems entirely designed around it so...


a common way to preserve concision on your end is to use the combination of namespaced keywords with namespace aliases, so even if you have a project-qualified namespaced keyword, you can use an ns alias


if i want to log the time a function takes, does anyone have a preferred method/framework recommendation


@candiedcode: clojure.core/time ( could be enough for simple checks, otherwise try Criterium


guys, what is the best book about clojure and web ??


I finished the joy of clojure, I would like to read a book about web and clojure.


cool thanks for sharing


@akiva: Thanks, that is the best ?


@mbrum, not only is it the best, I’m pretty sure it’s the only one.

Chris O’Donnell16:07:57

I'm having a bit of trouble properly constructing a lazy seq. I have a function f that pulls a list of things from the database. I tried (def aseq (apply concat (repeatedly f))), but the value of (f) seems to get cached and doesn't update with the database. Any ideas?

Chris O’Donnell16:07:45

To be clear, the intended behavior is, supposing (f) returns [:a :b :c], (:a :b :c :a :b :c ...), and if a new item :d is added, the seq will update to ... :a :b :c :d :a :b :c :d ....

Nicolas Boskovic16:07:44

How are you calling aseq?

Chris O’Donnell16:07:08

I think the problem is that lazy seqs are implemented using chunks. So even if I only take 10 elements, more than that may be realized.

Chris O’Donnell16:07:10

My lazy seq is eventually, but not immediately, updating to a new return value of (f).

Nicolas Boskovic16:07:35

That might be it

Nicolas Boskovic16:07:56

You'll have to eval the entirety of the return of f to check

Chris O’Donnell17:07:43

I'm working with a test database, so (f) just returns 1 or 2 elements.

Nicolas Boskovic17:07:54

And it's not updating at all? What does f do?

Chris O’Donnell17:07:52

;; (f) returns ["1"]
=> (def aseq (apply concat (repeatedly f)))
=> (take 2 aseq)
("1" "1")
=> (add-item-to-db "2")
=> (f)
["1" "2"]
=> (take 4 aseq)
("1" "1" "1" "1")
=> (take 6 aseq)
("1" "1" "1" "1" "1" "2")

Chris O’Donnell17:07:41

I want (take 4 aseq) to update immediately and give me ("1" "1" "1" "2")


@codonnell: lazy-seqs cache their realized values... after you took 4, the first four values are forevermore fixed


([& body])
  Takes a body of expressions that returns an ISeq or nil, and yields
  a Seqable object that will invoke the body only the first time seq
  is called, and will cache the result and return it on all subsequent
  seq calls. See also - realized?


@codonnell: if you are after a query-result-log of sorts, you could conj your results to a vector held in an atom

Chris O’Donnell17:07:38

@mccraigmccraig: First I took 2 elements, which become cached. Then I update the database, and so (f) evaluates to ["1" "2"]. It seems to me that the 3rd and 4th elements of the seq should be realized as "1" "2" when I do my (take 4 aseq), as they are not yet cached.

Chris O’Donnell17:07:30

What I'm after is a stream that cycles through the result of a database query. When one cycle finishes, the query is run again and we cycle through the updated result of my query.


ah, yeah, i see what you mean - iirc that's probably down to apply not being completely 100% lazy - you will have pain if you try and depend on exactly how many items have been realized from a lazy-seq

Nicolas Boskovic17:07:46

Why would you want a lazy seq then?

Nicolas Boskovic17:07:18

If you're going to be constantly running that it would make more sense to just have evaluated data

Chris O’Donnell17:07:32

@mccraigmccraig: yeah, apply could be the problem

Chris O’Donnell17:07:44

@nnbosko: I'm nto quite sure what you mean by "just have evaluated data"

Nicolas Boskovic17:07:10

I might have expressed myself incorrectly

Nicolas Boskovic17:07:02

Laziness isn't too useful in this particular use case

Chris O’Donnell17:07:35

Well the seq I'm creating is infinite; I'm not sure how I could construct it without laziness.

Nicolas Boskovic17:07:36

If you're going to have a sorts of query->eval->query loop you should probably handle the data eagerly


@codonnell: why not accumulate the query responses in a atom<vector> and then repeat the concatenation of those responses

Chris O’Donnell17:07:18

@mccraigmccraig: Once I take elements from the seq, I throw them away.


@stuartsierra: :artist/name it is indeed namespaced, but in lack of a better word, it is “explicitly” chosen for the problem at hand, not inherited from the ns you “happen” to implement the spec in. If my perspective is wrong, then should I go with ::name (eg. :myproject.something.artist/name), ::artist-name (eg. :myproject.specs/artist-name) or something else? More explicitly, I feel that I’m coupling my data to the implementation if I use the ”::” notation. I’m not talking about a library here, more like decoupled blocks (files) of code performing actions on some problem domain entity.

Chris O’Donnell17:07:09

that very lazy apply does look promising, though

Chris O’Donnell17:07:22

I need to step away right now, but I'll give it a whirl later when I have a bit more time


@mokr What matters is the context in which those keys are going to be used. If you know that the keywords in your database schema are only going to be used in your application, then it is safe to use “global” namespaces such as :artist/. If you intend to share that data with other applications which do not agree on a shared schema, then your keywords should be “namespaced” under your application.


The Datomic MusicBrainz example assumes the former.


@stuartsierra: Thanks, that confirms to me that I’ve gotten this right. The Datomic example seems like a good fit for the application I have in mind at the moment. The reason I asked is that (s/valid? …) kept returning true no matter what I gave as input or defined as spec using s/keys. Now I know that there is probably a bug in the code, not in my understanding. Thanks again!


The rather embarrassing bug was me typing :rec instead of :req in clojure.spec/keys.

Chris O’Donnell18:07:18

@mccraigmccraig: A slightly altered very-lazy-apply-concat worked for me. Thanks for the suggestion!


does clojure not end up evaluating let bindings if that binding is never used?


I have a strange case where I need to create a java object, then do some method calls on it, and I've encapsulated it in a 2 let bindings, but the method calls don't seem to be happening


because I end up returning the object


you might need a doall around the map


comp-config has nothing in it, but (first added-config) does. They are references to the same object though.


since map returns a lazy sequence that's never used, the anonymous fn won't be called


that makes sense


first forces the eval


thank you, that was it


Has an arity of 0 for atom been proposed? nil seems like the obvious default: (atom) #=> (atom nil)

Nicolas Boskovic19:07:57

you'd have to change jvm/clojure/lang/

Nicolas Boskovic19:07:10

plus the core.clj

Nicolas Boskovic19:07:16

or just the core.clj


Looks like a lot of hoops to jump through; maybe this weekend I'll try jumping through a few.


This idiosyncrasy of every-pred caused some confusion this morning:

user> ((every-pred (constantly false)))
user> ((every-pred (constantly false)) 1)
Is this a bug?


harold: looks intentional


hans: from the code, or some other piece of reasoning?


     (fn ep1
       ([] true)


In the implementation


No arguments = intentionally true


similar to

user> (and)


why not just call p with no args there?


It could be a closure.


Because predicates require an argument


You can't always call it with no args. e.g. even?


In that case it should throw...


(and would, if (p) were the code instead of true)



user> (and ((constantly false)))


That seems correct


Yeah, so this is still confusing to me:

user> ((every-pred (constantly false)))
No predicate returned a truthy value, but every-pred did...


I think I see what you're saying, and I think I agree that the ([] true) branch of code seems wrong.


You have to make a decision one way or another. No predicate was even executed, so claiming they returned either true or false is wrong either way.


Am I right in thinking that a predicate is any function that returns truthy/falsy, or is there a more rigid definition somewhere?


dg: in that case it seems weird for every-pred to just not execute its arguments... 😛


But it can't execute them. Your (constantly ...) is special in that it has a "no arguments required" version, but in general a predicate requires 1 argument.


So just (p) should throw.


Is that specified somewhere? The notion of a predicate requiring arguments?


It's a typical definition that a predicate has type signature p : X -> Boolean.


Zero-argument functions that produce boolean values can be interesting in the face of closures or mutable state. (defn before-noon? [] ...) for half-price before noon, etc.... (not super-purely functional, I know)


> Takes a set of predicates and returns a function f that returns true if all of its composing predicates return a logical true value against all of its arguments Ah I see. The way every-pred works is it calls each predicate with 1 argument at a time.


((every-pred even?) 2 1) does not translate to (even? 2 1), it translates to (and (even? 2) (even? 1))


The predicates are never invoked with anything other than arity 1.


oh, that makes sense.


Still seems like the ([] true) case could have been elided (if throwing were the intention) or just replaced with ([] (p)), ([] (and (p1) (p2)), etc.


Nice chat though, appreicate the thoughts danburtton, dg, and hans. 🙂

Nicolas Boskovic21:07:23

We have the best community, don't we, folks?


Again, we know for a fact that (p) should throw an exception, so it would be a mistake to do that.


A zero-argument function is not a predicate.


It's less the fact that it should throw an exception, and more a promise that it will never be called that way.


and a silent assent of truth, even in the face of (constantly false).


It seems like you could make a case that every-pred shouldn't have a 0-argument arity. Maybe it was added as a form of nil-punning when you apply a list to it?


Or, rather the function returned from every-pred shouldn't have a zero-argument arity.


It's sort of like asking "What's the product of no numbers?" In math that's typically defined to be 1.


The sum of no numbers is typically 0. 😉


That's the convention for addition, yeah


I see what you're saying about (and) then.

Alex Miller (Clojure team)21:07:54

@danburton I would not bother with a jira for (atom), I think that’s unlikely to get ok’ed

Alex Miller (Clojure team)21:07:48

@harold also see (+), (*), etc. and and every-pred are defined according to logic conventions for no 0-arity and that is intentional, not a bug.


Makes sense, and seems good for the empty-list punning case w/ apply that dg raised as well.


I'm still having a hard time understanding eduction. Do I have to call reduce on it?