Fork me on GitHub
#clojure
<
2017-02-02
>
seylerius00:02:06

Okay, say I'm trying to turn an options map into a seq of command line switches. {:wait true, :create false} would return an empty string, while {:wait false, :create true} would return ("-n" "-c"), to provide two examples.

seylerius00:02:20

What's the best way to do this? I'm thinking maybe map something that eats individual key-val pairs over the options map, but I'm not sure if there's something better than a nested cond and if to do it with.

bfabry00:02:58

(def opts->cli
  {[:wait false] "-n"
   [:create true] "-c"})

(remove nil? (map opts->cli opts))

bfabry00:02:12

iunno if that's too tricky... seems kinda neat

seylerius00:02:16

Ach. Won't match options that have arguments.

seylerius00:02:22

Like [:display "0:0"] or [:frame-params "ALIST"]

seylerius00:02:48

Not likely to need those, though.

seylerius01:02:25

Yeah, going to leave those out.

aengelberg02:02:23

Do take-last and drop-last have transducer equivalents?

joshjones03:02:22

@aengelberg due to the way reduce works, from left to right, and given how transducers work on arbitrary "streams" of input which can be channels, collections, network sockets, or anything else, I don't think there's really a concept of "the end". take yields the first n items. drop throws away the first n and then yields the rest. But it doesn't make much sense to work from the other end of the pipe for transducers. At least, I think I'm correct about this. I think the best you can do is to work off of a known collection size, and convert them as such: - for a list of size n, (take-last (- n x) v) yields the same list as (drop x v) - (drop-last (- n x) v) is (take x v) I know it's not what you want, but it might be the best you can do given the nature of how transducers work. I could be wrong of course, and would be interested to know how it can be done.

aengelberg03:02:47

Transducers can utilize the 1-arity to know when the end of a stream is.

aengelberg03:02:24

take-last could keep a buffer of the last N elements, and when the 1-arity is called, reduce all of those N elements in succession.

joshjones03:02:00

that might work -- have you tried anything?

aengelberg03:02:26

I haven't tried it, I was more asking to see if this has been officially implemented or do I have to roll my own.

joshjones03:02:13

this is pretty naive but works using transduce (does not work for into)..

(defn drop-last-xf [n]
  (fn [rf]
    (let [v (atom [])]
      (fn
        ([] (rf))
        ([result] (reduce rf (rf) (take (- (count @v) n) @v)))
        ([result input]
         (swap! v conj input))))))

(transduce (drop-last-xf 2) conj [1 2 3 4 5 6])
; => [1 2 3 4]

joshjones03:02:19

but it's definitely flawed as it doesn't work for an f other than conj

joshjones03:02:19

unfortunately there I've put the reducing function conj back into the transducer, which is what a transducer removes in the first place, which is why it fails

aengelberg03:02:05

if you just change reduce rf [] to reduce rf result that should make it work with into...

joshjones03:02:06

i'm going to change the above -- forgot i can call (rf) to produce initial value

aengelberg03:02:15

I think we can do better, i.e. we don't have to buffer everything in memory

aengelberg03:02:04

(defn alex-drop-last-xf
  [n]
  (fn [rf]
    (let [q (volatile! (clojure.lang.PersistentQueue/EMPTY))]
      (fn
        ([] (rf))
        ([result] (rf result))
        ([acc x]
         (if (> (count (vswap! q conj x)) n)
           (let [front (peek @q)]
             (vswap! q pop)
             (rf acc front))
           acc))))))

joshjones03:02:02

actually, i just realized we don't need state at all:

(defn drop-last-xf [n]
  (fn [rf]
    (fn
      ([] (rf))
      ([result] (reduce rf
                        (rf)
                        (take (- (count result) n) result)))
      ([result input]
       (if (vector? result)
         (conj result input)
         [input])))))

aengelberg03:02:31

your impls make an assumption of the type of the accumulator used in the reducer

aengelberg03:02:18

we don't know that the reducer takes and returns a vector

aengelberg03:02:40

that probably doesn't work on sequence which has an unusual stateful accumulator

joshjones03:02:45

the vector is just for accumulating the result until the last stage

joshjones04:02:42

can you find a case where the above doesn't work? it seems to work fine. the whole thing feels "wrong" but i'm not experienced with writing custom transducers so it's new to me

aengelberg04:02:29

;; with your latest version
user=> (sequence (drop-last-xf 2) [1 2 3 4])
()

aengelberg04:02:03

I stand by my version above, it seems to work with sequence and into and transduce

joshjones04:02:13

perfect! nice job

joshjones04:02:30

just happy i had a reason to try a custom transducer, was fun ๐Ÿ™‚

aengelberg04:02:33

I think where yours breaks down is that the 2-arity sometimes isn't actually used as an immutable accumulator

aengelberg04:02:48

so you have to keep your own if you really want to accumulate things internally

aengelberg04:02:09

agreed, custom transducers are a fun exercise ๐Ÿ™‚

aengelberg04:02:25

reading all of https://github.com/cgrand/xforms is a great way to expand your transducer-fu ๐Ÿ™‚

joshjones04:02:51

great, will have a look

aengelberg04:02:20

my attempt at take-last:

(defn alex-take-last-xf
  [n]
  (fn [rf]
    (let [q (volatile! (clojure.lang.PersistentQueue/EMPTY))]
      (fn
        ([] (rf))
        ([result]
         (reduce rf result @q))
        ([acc x]
         (when (> (count (vswap! q conj x)) n)
           (vswap! q pop))
         acc)))))

cgrand06:02:58

@aengelberg: you forgot to call the completing arity of rf on the result of reduce.

etherfuse06:02:02

curious if itโ€™s possible to have middleware in a compojure api only for a test environment. Iโ€™m playing with react native and canโ€™t write http request test because of CORS constraints. Once the app is compiled natively itโ€™s no longer a problem, and I donโ€™t want a vulnerability in a production server

etherfuse06:02:18

I canโ€™t write the test from the client, that is

aengelberg07:02:05

Thanks @cgrand, good point.

pesterhazy11:02:38

you're using assoc-in instead of assoc

tomaas11:02:34

@pesterhazy, using assoc also prints an empty result

pesterhazy11:02:26

1. test is a symbol from clojure.core - use another one

pesterhazy11:02:49

2. we're not seeing whole file

pesterhazy11:02:07

3. try restarting the whole repl

pesterhazy11:02:32

4. try peppering the file with println's to see if the functions are getting called

tomaas11:02:50

point 3 (restart the server) ๐Ÿ™‚

dacopare12:02:44

Does anyone have a recommendation for articles about creating DSLs using Clojure?

dacopare12:02:12

@mdib I've read that one, but I feel it stops too soon as it only implements a single function, echo.

dacopare12:02:01

There's also http://blog.oskarth.com/writing-a-dsl-in-clojure which is more about creating macros than a DSL.

dacopare12:02:18

The DSL I'm looking to create wouldn't have lispy syntax.

manutter5113:02:49

@dacopare are you familiar with Instaparse?

dacopare13:02:12

@manutter51 I've never heard of it.

manutter5113:02:47

Itโ€™s really a parser generator, but itโ€™s pretty nice, and kinda overlaps the concept of DSL

manutter5113:02:46

I mention it because you said you didnโ€™t want a lispy syntax, and Instaparse lets you define basically any syntax you like.

oskarth13:02:05

@dacopare: that's my article, was wondering why I was highlighted by slack :) what are you trying to do more specifically?

dacopare13:02:15

@manutter51: maybe it's time for me to learn about context-free grammars! ๐Ÿ˜… @oskarth: thank you for the article! My team have a partially-defined DSL, and I'm interested to see if I can parse and generate it using Clojure.

manutter5113:02:47

@dacopare Thereโ€™s a #instaparse channel too, not very active, but itโ€™s there.

oskarth13:02:06

I'd definitely look into instaparse for that kind of thing

oskarth13:02:40

Only used it briefly but it helps with a lot of the plumbing and has a nice declarative syntax IIRC

oskarth13:02:13

Maybe consider using clojure spec too

oskarth13:02:26

Can help with validation and generation+testing

tbaldridge13:02:03

@dacopare writing DSLs is a pretty broad topic. What sort of DSL is this?

tbaldridge13:02:19

@dacopare my two rules of DSL building 1) Don't. Use pure Clojure whenever you can. It's a language people already understand. DSLs are something people will have to learn. 2) Start with a data-defined backend. Don't use macros or instaparse, start with a solid data interface. Then add construction functions to build that data if needed. Finally add instaparse/macros that emit your AST. Now you have both a machine interface and ahuman interface

pesterhazy14:02:55

that's one hell of a talk description @val_waeselynck

dacopare14:02:34

@tbaldridge: the DSL exists independently of Clojure, I am just hoping that I can leverage Clojure to use it more easily.

dacopare14:02:05

Your second point has me slightly lost.

tbaldridge14:02:18

@dacopare so in Clojure we have lots of tools for working with data. So we want to base our DSL api of of data and functions. For example. Let's say your DSL has the ability to do this: "Y = X + 1". We could represent that as text (as I have done here), or we could represent it as machine-readable data:

tbaldridge14:02:14

{:op :assign
  :left {:op :var :name 'Y}
  :right {:op :+
                 :a {:op :var :name 'X}
                 :b {:op :const :val 1}}}

tbaldridge14:02:45

That's all hashmaps, you can use get-in and update-in to modify them, and you could in about 10 minutes write a multi-method based interpreter for that.

tbaldridge14:02:34

Now writing those hashmaps is a pain for humnans, so you could wrap it up with functions and macros: (assign :left (var Y) :right (add X 1))

tbaldridge14:02:50

And those functions could emit the hashmaps that are passed to your multi-method interpreter.

tbaldridge14:02:44

And finally, after all that, sometimes using Clojure is a pain for some users, so that's when you bring out Instaparse and parse "Y = X + 1" and have that use your functions or somehow emit the hashmaps that are fed into the multi-method interpreter.

arnaud_bos15:02:40

I'm right in the middle of this talk's universe right now, this hurts so much...

roelof15:02:08

Any recommendations for a nice free weather api so I can make a site where the weather forecast and current weather is shown

dacopare16:02:21

Thank you @tbaldridge, I'm very happy working with Clojure datastructures and multimethods. It was the transformation of this into a string and vice versa that I really meant to ask about. I'll have a look at instaparse.

aengelberg16:02:29

#instaparse may not be very active but I'm fairly responsive if someone has a question for me :)

aengelberg16:02:25

Instaparse is good at parsing, it doesn't have great capability to "unparse" or generate a string from data.

aengelberg16:02:29

But I like @tbaldridge's point about leveraging Clojure or EDN for your DSL, not strings, if you can. In which case you don't need instaparse

cap10morgan17:02:58

Would Clojure ever consider setting the UNICODE_CHARACTER_CLASS flag to true in all of its regexes? Would be really nice to not inherit Java's Unicode brokenness there (so, for example, #"\w" will match chars like รฑ).

ghadi17:02:31

cap10morgan: it would potentially change people's code in the wild, so probably not

ghadi17:02:34

(even if changing a supposedly broken change, Rich and core team value compatibility)

cap10morgan17:02:50

Yeah, understandable.

bja17:02:01

having your dsl as edn also allows automated upgrades to future dsl versions. This is useful if you create and store a bunch of random snippets.

tjtolton18:02:26

whats the function im thinking of? given a nested map

{:a {:b 1} :c {:d 2}
I want a sequence of
[[:a :b 1] [:c :d 2]]

tjtolton18:02:05

so essentially the "paths" of the leaf values of the nested map

tjtolton18:02:34

a forloop is pretty easy if its a known level of nesting, but I feel like theres a simple, perhaps even a core function im overlooking

bostonaholic18:02:44

donโ€™t you mean [[:a :b 1] [:c :d 2]]

joshjones18:02:45

do you mean [[:a :b 1] [:c :d 2]] or am I misunderstanding?

bfabry18:02:01

someone was asking about how to do this with specter in #specter the other day

tjtolton18:02:03

thats what i meant

hiredman18:02:21

(fn f [path item] (if (map? item) (for [[k v] item i (f (conj path k) v)] i) [(conj path item)]))

bfabry18:02:21

oh but I can't search for those messages because slack. damn

pesterhazy18:02:47

There is a log service

hiredman18:02:36

user=> ((fn f [path item] (if (map? item) (for [[k v] item i (f (conj path k) v)] i) [(conj path item)])) [] {:a {:b 1} :c {:d 2}})
([:a :b 1] [:c :d 2])
user=> 

tjtolton18:02:06

yeah, that looks right! lol. sorry, was thinking there was a core function or something

hiredman18:02:07

not that I am ware of, I've written some variation of that a lot. maybe it could be turned in to a generic tree processing function like tree-seq

nathanmarz18:02:46

@tjtolton @bfabry @hiredman with specter:

user=> (use 'com.rpl.specter)
nil
user=> (def PATH-MAP-WALKER
  #_=>   (recursive-path [] p
  #_=>     (if-path map?
  #_=>       [ALL (collect-one FIRST) LAST p]
  #_=>       STAY
  #_=>       )))
#'user/PATH-MAP-WALKER
user=> (select PATH-MAP-WALKER {:a {:b 1} :c {:d 2}})
[[:a :b 1] [:c :d 2]]
user=> (transform PATH-MAP-WALKER str {:a {:b 1} :c {:d 2}})
{:a {:b ":a:b1"}, :c {:d ":c:d2"}}

bja19:02:21

has anyone experienced (require 'potemkin.collections) failing the first time and then succeeding the second time? (on [potemkin "0.4.3"])

plins19:02:13

hello everyone, sorry if this is a stupid question, i'm new to clojure & jvm .. Im building some microservices and I would like to encode the response with gzip. is there any standart way to do it with compojure-api/compojure/ring ? searching google ive found only code snippets, but it is possible to configure the framework to do this?

ghadi19:02:06

there are various gzip middlewares for ring, but normally most people delegate to nginx to do things like TLS termination and gzip serving

plins19:02:02

thx @ghadi , found some of them in google, I was using bad keywords for the search ๐Ÿ™‚ unfortunatly ngnix isnt a option in this case

bja19:02:03

you might also try configuring gzip using whatever underlying ring adapter (undertow, jetty, etc)

bja19:02:57

also, I've traced my problem to an issu ewith [metrics-statsd "0.1.8"] (in case my previous question worried anyone)

reitzensteinm20:02:16

@plins you said it's for a microservice, is it going over your internal network?

plins20:02:11

well, all of our infrastructure is hosted on AWS, and a php service will call the clojure one (still dealing with legacy code :~ )

plins20:02:35

@reitzensteinm , so its going on our internal network i guess

grav21:02:05

Anyone using VS Code with the Continuum extension?

roelof21:02:12

someone who has uses ez-web for pagination and wants to share the code so I can look how I can make things work

mike_ananev21:02:18

hi! I have some confusion working with inst in Clojure. It is java.util.Date object. Current Java 8 introduced new classes to work with time and java.util.Date became deprecated. Is there any plans to add support of new java classes for date and time?

tbaldridge21:02:46

I doubt it as it would probably break existing code. For reading data though, you can add your own handlers to parse into new date types

mike_ananev21:02:38

@tbaldridge thanks. Already implemented my own handlers, but it looks a bit rough

sova-soars-the-sora22:02:36

Hi I'm wondering how to append to a vector of maps in an atom.

sova-soars-the-sora22:02:10

(swap! atom-name [:vector 0] {:new "map}) replaces the first map in the vector of maps....

jr22:02:30

(swap! atom-inst conj new-map)

schmee22:02:31

user=> (def a (atom []))
#'user/a
user=> (swap! a conj {})
[{}]

schmee22:02:21

does anyone have a collection of important/well-written papers in persistent data structures?

schmee22:02:41

preferably recent (i.e. post-Clojure) developments

schmee22:02:08

Iโ€™m curious about what the state of the art is, and if/how clojures core data structures could be improved

dpsutton22:02:59

there was a conj talk from seattle about this very topic

dpsutton22:02:27

sorry clojure west

dpsutton22:02:48

excellent talk there^

schmee22:02:29

hah, Iโ€™m watching that very talk right now! ๐Ÿ˜„

schmee22:02:43

thatโ€™s what made me pose the question ๐Ÿ˜„

sova-soars-the-sora22:02:50

Hrm... I want to conj into a nested vector... it has a keyword :blurbs ... How would I specify which sub-set of atom data to conj into?

dpsutton22:02:07

i've looked on arxiv before but didn't notice too much

dpsutton22:02:15

but i don't know how to navigate that place too well

sova-soars-the-sora22:02:31

(def atom ({:top [{:id 1} {:id 2} {:id 3]}))

sova-soars-the-sora22:02:45

how would one add a hypothetical {:id 4} in that fashion?

sova-soars-the-sora22:02:01

Yours truly, perpetual_beginner

tolitius22:02:32

dev=> (def a (atom {:top [{:id 1} {:id 2} {:id 3}]}))
#'dev/a
dev=> (swap! a update :top conj {:id 4})
{:top [{:id 1} {:id 2} {:id 3} {:id 4}]}

sova-soars-the-sora22:02:48

Awesome! Thank you ๐Ÿ™‚