Fork me on GitHub

skrat: i love your user pic


I'd really like to get a stable checksum/hash of an arbitrary persistent data structure, has anyone done this?

(defn canonical-data-structure [build-config]
  (walk/prewalk (fn [x]
                     (map? x) (into (sorted-map) x)
                     #_(string? x) #_(.hashCode x)
                     (set? x) (into (sorted-set) x)
                     :else x))

(defn canonical-hash [data]
  (-> data


It hasn't been working for me.


I need an algorithm or method that can calculate missing values from time series. Basicly I have a dataset with hourly readings from energy meters and I need to calculate the hourly energy usage. The challenge is that somestimes alot of data points are missing. example:


any tips ? maybe Incanter?


make a record with slots for hour, interpreted/recorded, meter-total, usage, then just loop over, storing all records without a recorded meter total until you hit the next one with a metered result, and then just put a linear interpolation across all the ones without readings


if this makes it up to the current hour, then you are obviously stuck and can make a decision as to leave it without measurements or to estimate based on the last three or four known points


oh wow. didn't know you were missing so much data


maybe loop twice


first loop create a trend, and then have records for each reading as :deviation-from-trend and :predicted-value


in this way you can create your model, allow for changing needs over the day, ie, noon usage isn't the same as early morning, etc.


and you fill in each value and then when you need to consume you just grab (or (:metered-reading row) (:predictive-value row))


@dpsutton: thank you very much! Was reading about gaussian kernel smoother on wikipedia and my head was about to explode.


perfection is the enemy of good


sorry, perfection is the enemy of done


don't over complicate it


come up with a super simple model


get that model from the data and then map its predictions onto the data


bam problem solved


then you can in the future increase the complexity of the model


but that's a separate and orthogonal problem from the one at hand


simple programming problem, complicated modeling problem


you might be able to solve your woes with this one.


i had never heard of that library. seems chock full of stuff


very cool! thanks again


looks reasonable I think


Hi, say I'm do a bunch of checks like:

(or (= something "foo")
            (= something "bar")
            (= something "baz"))
Is there a nicer way of doing it? Doing it like this doesn't feel right in Clojure.


(contains? #{"foo" "bar" "baz"} something)


... I knew that too... Happy Sunday morning 🙂


Trying out clojure.spec — and I'm having trouble with some of my maps. I want to validate maps of the form {:insert [{} {} {} {}]}, where keys are (s/def ::op #{:insert :replace :delete :state}). How can define a type ::data that will validate a map with (optional and non-namespaced) keys conforming to ::op?


I realize I can list those explicitly in the :opt-un vector passed to s/keys, but that isn't quite satisfactory — and besides all operations have the same parameter structure, so I would like to later also validate their content without defining the same spec for each op.


hi folks - i have a clojure project in cursive that has become fantastically slow to boot a REPL. everything appears to have parsed and executed in under 10 seconds but then clojure spends the next 50 seconds or so compiling somethingorother and often timing out


...what is clojure so busy compiling? the code has already executed...


and things are garbage collection like mad in the meantime too

Alex Miller (Clojure team)12:06:17

@jrychter: why is :opt-un not satisfactory? you can reuse the same spec for each attribute too

(s/def ::op-val ...whatever...) ;; define the content of an op here
(s/def ::insert ::op-val)
(s/def ::replace ::op-val)
(s/def ::delete ::op-val)
(s/def ::state ::op-val)
(s/def ::command (s/keys :opt-un [::insert ::replace ::delete ::state]))

Alex Miller (Clojure team)12:06:54

@benzn if you can repro outside of Clojure, you might try taking thread dumps of the process (ctrl-break in win, ctrl-\ in *nix, or kill -3 the process, or use jstack) to see what it's doing in that extra time.


@alexmiller: I tried that, and somehow thought it was too verbose, I thought I could just list all the ops in a single set. But having looked at it again, it doesn't look so bad…


@alexmiller: ok, now I remember. The next level in my data structure contains a table name (as a keyword). So you could get an {:replace {:stock {more-data here}}. So there again, instead of my current (def ^:const +watched-tables+ #{:stock ...}) I will need to declare each of them in two places. I guess my data structures don't really fit with clojure.spec, and I should probably change them to have the op and table names as values, not as keys…

Alex Miller (Clojure team)12:06:22

@jrychter: you might want to declare the next level down using something like map-of, since these are not fixed keys


@alexmiller: thanks. The more I think about it, the more I come to the conclusion that I should have a fixed key structure and keep data in values, which would let me use multi-specs. With the added advantage of them being extensible (doesn't make sense for ops, but does for tables).


clojure.spec question, it seems you can you s/or in place of s/alt just fine. Are there cases where this doesn't work? Why do we need both?


(s/conform (s/cat :s string? :x (s/alt :n number? :k keyword?)) ["foo" :bar])
;;=> {:s "foo", :x [:k :bar]}

(s/conform (s/cat :s string? :x (s/or :n number? :k keyword?)) ["foo" :bar])
;;=> {:s "foo", :x [:k :bar]}


About argument order in clojure. I have posted some comments in!topic/clojure/iyyNyWs53dc, but unfortunitly no reply. Can some one give some comments here?


To partially answer my question, s/or generates a clojure.spec.Spec object, while s/alt returns a regex operator (represented as a map and used by the spec regex machinery), so I suppose these compose differently


(s/alt :n number?)
;;=> {:clojure.spec/op :clojure.spec/alt, :ps (#function[clojure.core/number?]), :ks (:n), :forms (clojure.core/number?), :id #uuid "72ae2d04-43ef-4e33-a8c2-e3ca70cc5844"}
(s/or :n number?)
;;=> #object[clojure.spec$or_spec_impl$reify__11565 0x1a6b7806 "[email protected]"]


still would love to see an example where the result differs, I tried conform, valid?, exercise, explain, and at least for the basic case above alt and or always behave the same


In case anyone else is interested, this snippet illustrates the differences 🦆


@plexus: I was wondering about this, thanks!


yeah it makes total sense if you keep reminding yourself it's just regexes. I'm not used to applying that to anything that isn't a string 🙂


Hm. I think I might be fuzzy on this still. What is difference between specs and regex ops that causes '(:bar :baz) to be consumed by or but not by alt in the second conform above?


Because alt is a regex operator you stay on the same "level", in the way that #"a(bc|d)" matches "abc", (s/cat :a keyword? :x (s/alt :s (s/cat :b keyword? :c keyword?) :d keyword?)) matches [:a :b :c]


in other words, it doesn't match a nested structure, it just matches a flat seq


if you do want to match a nested structure like [:a [:b :c]] you would add an s/spec to start a new spec that matches the nested sequence


because or is not a regex operator but its own type of spec, it implicitly causes such nesting


don't know if that makes sense... it's a bit tricky to explain well


to put it another way, as long as you're only using regex operators (cat, +, *, ?, alt), you're matching a single flat sequence.


Each item in the sequence can be validated/conformed by a predicate or a spec (registered or inline)


if the item is itself a sequence, you can express that with a spec (s/or, s/spec) or a predicate (vector?) inside the regex


@plexus: great explanation. Thanks, this definitely clears it up.

Alex Miller (Clojure team)15:06:25

We're typically used to regexes matching characters in a string. in this case though we are matching Clojure data in a sequence. It's not possible for a single character in a string to also be a substring, but it is possible for an element in a Clojure sequence to be another nested sequence. That's what introduces this complication here.


right @alexmiller, well put!


In a multiple process situation, say I want to synchronize around a system resource (a directory) and guarantee a release of that resource on process exit. Is there a good Java-y way to do that? File Locks seem too brittle I'd love to have something that just won't survive the end of a process.


Maybe it is too much over-engineering but, hazellcast offers a distributed lock


it is pretty easy to use with java interop...


yeah I really don't want to bring in any more deps


this is for figwheel


thanks though

Alex Miller (Clojure team)16:06:50

@bhauman you can use Java finalizers for that although there are whole talks worth of caveats about them

Alex Miller (Clojure team)16:06:39

there are also ways to create temp files and mark them to be deleted on exit

Alex Miller (Clojure team)16:06:54

can't say I've found that to actually work reliably though


reliability is the caveat


it seems much more reliable that manual using of files for locking


deleteOnExit can some sometimes fail as well, would love something like opening a socket, you know its going to go away after the process exits, but sockets don't have arbitrary names

Alex Miller (Clojure team)16:06:37

I'd trust something like FileLock more than the others I mentioned

Alex Miller (Clojure team)16:06:13

but I haven't used it to know how reliable it is


later, also maybe considered strange option, but if you have postgresql already a dependency in your project, you can use an advirsory locks to hold a lock. It works much more in a "socket way" if connection is closed, lock is released.


FileLock does look like the best of the options


Is clojure/algo.generic common/idiomatic or is there a better way to get fmap? I’m torn between just implementing fmap myself and using it


I care in particular for maps; I want {k v} => {k (f v)}


(defn ^:private mapvals
  [f m]
  (reduce-kv (fn [m k v] (assoc m k (f v))) (empty m) m))
is not very ugly I suppose:)


@lvh Medley is one of the more common utility libs that has map-vals


Ah; thanks — I had seen it before but forgot about it


there's another commonly used utility library that also has it but can't remember what it's called now, and it seems is still down 😞


I've been using my own transform-v and transform-kv since Clojure 1.0, and I've always wondered why they aren't in core. It seems everybody implements their own version…


there will always be a Most Useful Function Not In Core


True, but these two (especially transform-v also known as fmap) are the ones I encounter pretty much everywhere. I recently discovered three different versions within our company, written by different people over the years.


I made a screencast to introduce people to visualising their data in proto-repl-charts:


I've always been bothered by the use of namespaced keywords as sentinels


likewise -- I tend to use (Object.) or (reify)


bronsa: (reify) is cute -- I guess it's ostensibly more portable than (Object.)?


Is it possible to return keys from an execute! in


is there a preferred library/framework for writing integration tests for compojure api routes?


I have a Dockerfile modelled after the example in Clojure's docker hub and any time I change the project.clj, it seems to download all maven dependencies again. Is that by design?


cddr: hard to avoid I'd think


I've mitigated that some by linking in my ~/.m2 but that doesn't work when building images, just when running stuff in the container


Yeah I can definitely see why it would be. Just wanted to check I wasn't doing something silly. It's only really a problem in the early stages of a project when you keep adding new libs. And even then it's not too bad


I could imagine some sort of fancy programmatic modification of the Dockerfile used in tandem with leiningen's local-repo feature


@james: That looks lovely - proto-repl is looking really nice.