Fork me on GitHub
#beginners
<
2020-02-10
>
jwoods00:02:49

Is there a way to read in a csv and assign the headers or first row to create a map with the headers as keys?

practicalli-johnny11:02:25

Clojure.data.csv is usually used to transform CSV data to Clojure data structures. https://github.com/metasoarous/semantic-csv parses CSV data and allows you to do more specific transformation and conversion.

bartuka01:02:02

@jdwoodsy1 in the README file of https://github.com/clojure/data.csv has a section about "parsing into maps"

Matthew Pettis01:02:38

I have a list of functions (f g h) and values (1 2 3) and I want to get a sequence of values of evaluating the function to the argument of the same index, as in: ((f 1) (g 2) (h 3)) I thought this would work: (map #(%1 %2) '(str inc) [1 2]) but it does not. I ended up doing this: (map #(eval (list %1 %2)) '(str inc) [1 2]) ("1" 3) This seems overly verbose, and that I am missing something simple. Is there a more idiomatic way to do this?

Cameron02:02:00

(map #(% %2) [str #(* % 2)] [1 2])
=> ("1" 4)
It looks like you just numbered your first arg wrong

andy.fingerhut02:02:20

user=> (map (fn [f arg] (f arg)) [inc dec +] [1 2 3])
(2 1 3)

andy.fingerhut02:02:53

#() syntax is nice in the majority of cases when it works, but sometimes just using (fn [ ...] ...) can make it easier to figure out how to make it work correctly.

Matthew Pettis02:02:17

Thanks both. One of the things I tried that I didn't post and didn't work was using a list of symbols for the functions rather than a vector of function (symbols?):

Matthew Pettis02:02:34

(map (fn [f arg] (f arg)) '(inc dec +) [1 2 3])

Matthew Pettis02:02:26

converting to a vector gives me nils as well: (map (fn [f arg] (f arg)) (vec '(inc dec +)) [1 2 3]) (nil nil nil)

Matthew Pettis02:02:18

So I think my question refines down to "what is the difference between a vector of functions and list of quoted symbols in this case?"

seancorfield02:02:08

Resolution of the symbol to the actual value.

Cameron02:02:19

I'd like someone to confirm though ahaha

seancorfield02:02:36

@matthew.pettis

user=> (map (fn [f arg] ((resolve f) arg)) '(inc dec +) [1 3 5])
(2 2 5)

👍 4
🚀 4
bartuka02:02:31

nice, I was trying to avoid the evalbit

Matthew Pettis02:02:54

Thank you @seancorfield --- I haven't seen resolve...

seancorfield02:02:10

'inc is the symbol named inc. inc (no quote) will be evaluated by Clojure -- resolved to the symbol's current bound value. In this case clojure.core/inc is bound to a function (value) that increments its argument.

👍 4
Matthew Pettis02:02:24

That makes the #() formulation work too: (map #((resolve %1) %2) '(inc dec +) [1 2 3]) (2 1 3)

seancorfield02:02:47

Yes, #() is just shorthand for (fn [,,,] ,,,)

seancorfield02:02:26

Back to '(x y z) vs [x y z] -- the former is a quoted list so it is not evaluated; the latter is a vector of values so the symbols are looked up (resolved). '[x y z] is also quoted and not evaluated.

bartuka02:02:37

I'm not into macros yet, but I have a question for this specific problem, would be easier to handle this list of symbols inside a macro? even avoiding the resolve call?

seancorfield02:02:50

(list x y z) would evaluate the elements, resolving the symbols to their bound values.

seancorfield02:02:56

@iagwanderson macros just rewrite code to code - they don't evaluate anything (they are executed at "read" time, before code is compiled and then executed).

bartuka02:02:34

yeah, I changed my eval for resolve and it looks like the same as the map version

(defmacro custom-map [fs args]
  (list 'map '(fn [v arg] ((resolve v) arg)) fs args))

seancorfield02:02:31

You don't need a macro for that.

seancorfield02:02:00

The first rule of macro club is: don't use macros.

👍 8
Matthew Pettis02:02:23

For completion, that also explains why I was getting nil values out. I was evaluating things like ('inc 2) which yields nil. I don't understand why having a "symbol named 'inc'" in the function postion in a list passed to the evaluator returns nil though...

seancorfield02:02:05

Because it looks itself up in its argument, as if it were (get 2 'inc) which returns nil.

seancorfield02:02:37

user=> ('inc {'inc 1 'dec 2})
1

👍 8
seancorfield02:02:24

user=> ('inc 2 :not-found)
:not-found

seancorfield02:02:40

user=> (get 2 'inc :not-found)
:not-found

seancorfield02:02:33

Keywords and symbols look themselves up in their argument, in the same way that maps and sets look their argument up inside themselves. It makes working with associative data structures very easy.

Matthew Pettis03:02:23

So, to state in my own words: inc is the name of the symbol 'inc. In the sexp type into the repl (inc 2) In the repl, you use the name, inc. The name will resolve to the symbol 'inc.

-----------------
| name | symbol |
-----------------
|  inc | 'inc   |
-----------------
That symbol 'inc is associated with a function val in a var:
------------------------------
|            var              |
------------------------------
| symbol |        val         |
------------------------------
|  'inc  | #'clojure.core/inc |
------------------------------
And then, the function at #'clojure.core/inc is put into the function position of the Abstract Syntax Tree, and 2 is put into its first argument position, and then evaluated. Is that correct? Are there two levels of indirection going from the text in the repl -- looking up the symbol from the name inc, and looking up the function from the symbol 'inc?

hiredman03:02:38

No, the symbol is name

hiredman03:02:26

'inc is a short hand in the reader for (quote inc) which is a list of two symbols

Matthew Pettis03:02:31

Thanks for this... this seems different from Sean's explanation of 'inc as symbol, and inc as name...

hiredman03:02:53

The evaluator (the compiler) says the meaning(the result of evaluating) of (quote inc) is the symbol inc

hiredman03:02:35

Treating these thinks precisely takes a lot of care and it is very common for experienced clojure programmers to gloss over the details because everyone knows what you mean

Matthew Pettis03:02:21

Makes sense. These questions are helping clarify this a bit, I think...

hiredman03:02:49

When Sean said 'inc is the symbol named inc what he meant was the evaluator evaluates 'inc to the symbol inc

Matthew Pettis03:02:30

ah, ok, that helps. inc is the symbol then...

hiredman03:02:14

Quoting is a general facility that can be applied to anything you pass to the evaluator

hiredman03:02:10

(+ 1 2) evaluates to 3, but '(+ 1 2) evaluates to the three element list (+ 1 2)

Matthew Pettis03:02:36

and + is a symbol, not the name of the symbol...

Matthew Pettis03:02:47

Thanks. I thought I had symbols understood until I encountered the concept that symbols have names, which is not the same as the symbol itself...

hiredman03:02:03

The evaluator either resolves symbols to a var (something def'ed) or a local (something let bound or a function parameter)

hiredman03:02:27

Like, technically, symbols do also have names, the name of the symbol inc is the string "inc"

Matthew Pettis03:02:37

Now, the var has a value, correct? Like the symbol inc maps to a var that has a value, which is the function that increments a number. If I "rebind" the symbol inc, do I point it at a different var, or do I stick a different function in the value of the var? Or is that non-sensical?

hiredman03:02:26

Symbols are technically composed of a namespace part and a name part. So the symbol inc has the name "inc" and the namespace nil while the symbol clojure.core/inc has the name "inc" and the namespace clojure.core (the namespace part of a symbol is itself represented as a symbol)

hiredman03:02:16

When the evaluator encounters a symbol it from context figures out if the symbol is the name of a local (let bound, function parameter) or needs to be resolved (again using some context) to a var(something defined using def or some derivative)

hiredman03:02:54

A var is sort of a little mutable cell, like a 1 element array, so you can override it's value, or you could just have a local, the exact behavior will depend on what you mean by rebinding

Matthew Pettis03:02:59

ok, thanks. In your previous paragraph, then, for the symbol inc with the namespace of nil, is that the case when inc is the name of a local?

hiredman03:02:23

That will depend on context

hiredman03:02:32

(let [inc 1] inc)

hiredman03:02:50

That inc is a local, because of the surrounding let

Matthew Pettis03:02:19

ok, makes sense, you look to see if a symbol has a local binding, and if so, use that value. if not, you have to walk up to enclosing environments to find something it is bound to (a namespace, or possibly another local binding that encloses the first one)...

hiredman03:02:34

more or less

hiredman03:02:22

A wrinkle here is clojure is the evaluator compiles to jvm byte code and them hands the byte code to the jvm to run. All the symbol resolution stuff the evaluator does happens at compile time

Matthew Pettis03:02:39

To the above statement that inc has a name of "inc" and a nil namespace, and the symbol clojure.core/inc has the name "inc", then when I use inc in the repl, how does it know to choose the clojure.core namespaced one, and not the first one?

Matthew Pettis03:02:53

Thanks for your time here... I think I need some paper that lays this out somewhere, I'll see what I can find. If you have a resource that lays this out somewhere, I'd be happy to read it...

andy.fingerhut03:02:53

When reading the text of the expression in the REPL, the Clojure reader gets a symbol, and a number, which you can check yourself like this:

user=> (def tmp (read-string "(inc 2)"))
#'user/tmp
user=> (map class tmp)
(clojure.lang.Symbol java.lang.Long)

👍 4
Matthew Pettis03:02:00

A bit confused... from above, if inc is the name, how does the reader get the symbol 'inc? Assuming I am interpreting that correctly. I assume if inc is the name, there has to be some lookup path from name -> symbol -> var value...

andy.fingerhut03:02:56

Doing eval on a symbol looks up a table of names in the current namespace (or in the namespace that is the prefix in the symbol, e.g. foo.bar/inc), e.g.:

user=> (eval 'inc)
#object[clojure.core$inc 0x426b6a74 "clojure.core$inc@426b6a74"]
user=> (class (eval 'inc))
clojure.core$inc
user=> (instance? clojure.lang.IFn (eval 'inc))
true

andy.fingerhut03:02:09

When Clojure is given a list in parentheses to evaluate, without a leading ' , and the first element does not name a macro, it evals all elements of the list, including the first, then invokes the function that is the value of the first expression.

Matthew Pettis03:02:59

I'm getting closer. One thing that seems odd is that the following is equivalent then:

Matthew Pettis03:02:07

sess3=> (eval 'inc)
#object[clojure.core$inc 0x7fbdb403 "clojure.core$inc@7fbdb403"]
sess3=> (eval inc)
#object[clojure.core$inc 0x7fbdb403 "clojure.core$inc@7fbdb403"]

Alex Miller (Clojure team)03:02:21

functions (like most things) evaluate to themselves so the second eval returns the same thing

Alex Miller (Clojure team)03:02:37

quote means "read, but don't evaluate"

Alex Miller (Clojure team)03:02:52

so 'inc is read (as a symbol), the end

Alex Miller (Clojure team)03:02:34

inc is read as a symbol, then evaluated. in general, everything in Clojure evaluates to itself, with the with the two main exceptions: symbols and lists

Alex Miller (Clojure team)03:02:54

symbols, evaluate to what they refer to in the scope of the current namespace

Alex Miller (Clojure team)03:02:08

(lists evaluate args, then invoke the first element)

Alex Miller (Clojure team)03:02:22

and now you know 90% of the Clojure evaluation model

Alex Miller (Clojure team)03:02:35

(macros being the next 9%)

Matthew Pettis04:02:30

Thanks! In a side channel, I'm working through the concept of a symbol vs. the name of the symbol, which I keep tripping on...

Matthew Pettis04:02:49

(is that the last 1%?)

Alex Miller (Clojure team)04:02:19

symbols are names the way I think about it

Alex Miller (Clojure team)04:02:53

(the last 1% is all the other special cases which we won't talk about :)

ScArcher04:02:31

I'm curious about best practices around calling APIs with Clojure. The code looks nice and elegant when just making the http request, but I'm not sure where to put the "parse the resulting json" code and the "http error handling" stuff.

ScArcher04:02:13

It looks like clj-http will automatically handle the json parsing, so now it's just a question of best practices on error handling.

jysandy08:02:56

I typically use a cond to check the HTTP status code and handle the error as appropriate. This usually means monitoring/reporting of some sort.

Matthew Pettis04:02:23

I'll go back and re-read, but I read through "Joy of Clojure", and I don't have it here in front of me, but I recall the mention of names of symbols as distinct from the symbols, and I think the material difference was that you could associate metadata with one but not the other. I don't recall which one...

Alex Miller (Clojure team)04:02:01

you can associate metadata with symbols

skoude06:02:47

if I have a vector of maps like: [{:firstname "test", :lastname "test"} {:firstname "test1", :lastname "test2"}] how can I get a new vector of maps with only the firstname in map ?

seancorfield06:02:46

(mapv #(select-keys % [:firstname]) data)

seancorfield06:02:10

Or do you mean (mapv :firstname data) to get just a vector of first names?

skoude06:02:27

thanks @seancorfield, this was just what I was looking for,. Just switching to clojure and had been figuring out this for a while now 🙂

4
jakuzure08:02:37

Hi, using the carmine library for redis I wanted to scan and match some keys and found out I need to use the accumulator to get the whole output, seems like it's already implemented in carmine, but I don't know how to use it: https://github.com/ptaoussanis/carmine/blob/master/src/taoensso/carmine.clj#L750 In particular I don't understand what rf refers to, can anybody help?

Lennart Buit08:02:59

rf usually refers to a reduction function. So a function that takes the accumulated value, and the current, and produces a new accumulated value

jakuzure08:02:27

So I need to provide my own function? I thought carmine would take care of that and I just give it my scan function, I'm a bit lost right now 😅

metehan10:02:46

hi people I am live streaming my work. I am not advanced clojurescript guy but might be usefull for newbies

seancorfield18:02:59

@UHK8B8STX I find the end result much harder to read. I think let-map tries to be far too clever.

seancorfield18:02:46

There are some libraries out there that have some variant of a "locals" macro for constructing hash maps from local bindings which is the core of what this is doing, but without all the destructuring magic.

mloughlin19:02:55

&env in the wild! Very good. I like the way local-map gives control back to the caller, instead of wrapping let.

mloughlin19:02:00

That's something I've been trying to aim for in my own code.

Alex Miller (Clojure team)14:02:49

imo, the code before the macro was easier to read

Alex Miller (Clojure team)14:02:29

maybe working more on the signature of some-magic-parsing-fn function would have been more profitable than correcting the results, not sure

Alex Miller (Clojure team)14:02:59

my general suggestion would be - don't be too quick to dismiss obviousness :)

mloughlin14:02:56

my initial feeling was let-map feels like something out of the CL world

mloughlin14:02:00

but that's more about the interface of the macro - overloading (let [x y]) without having a body using the bindings. If there was a map literal declaration that let me access previous values, I'd probably use it here.

FiVo17:02:07

Is there a way to access the forms in a clojure.lang.ReaderConditional? So in the result of something like (read-string {:read-cond :preserve} "#?(:cljs (+ 1 2) :clj (+ 2 3)))")

ghadi17:02:24

@finn.volkel you can call :form on it

👍 4
FiVo17:02:31

@U050ECB92 Is there a way to obtain this information generally for clojure objects or you looked at the clojure source code?

Alex Miller (Clojure team)17:02:38

in this case, no there is not an easy way to determine this without being told as the object implement lookup, but not the rest of the map contract

ghadi17:02:01

in this case supers on the reader conditional, and saw it was an ILookup

ghadi17:02:06

then docs weren't helpful

ghadi17:02:11

so I grepped the source

Alex Miller (Clojure team)17:02:54

in this case, the ReaderConditional object is really a thin wrapper around a map-ish object with lookupable :form and :splicing? (a flag)

Alex Miller (Clojure team)17:02:39

I don't honestly remember now why we made an object for this rather than just making a record or something, which seems like it would have been sufficient

ghadi17:02:41

doesn't it need to be instantiated by LispReader?

Alex Miller (Clojure team)17:02:57

yeah, that might have been it

teslanick17:02:42

I’m trying to wrap my mind around transducers — what’s the difference between sequence and eduction? Reading the source code didn’t help that much 😕

ghadi17:02:31

@teslanick common: they are both contexts where a transducer can be applied. Both presume that the source collection is at least iterable different: sequence caches realization of the xf+collection, eduction does not

ghadi17:02:55

(defn less-promises-eduction [xform coll]
  (reify clojure.lang.IReduceInit
   (reduce [_ f init]
     ;; NB (completing f) isolates completion of inner rf from outer rf
     (transduce xform (completing f) init coll))))

ghadi17:02:27

one could imagine a variant of eduction that didn't require anything of the underlying collection except that it is reducible

ghadi17:02:34

(AKA dropping the iterable requirement)

ghadi17:02:00

which would differentiate sequence and eduction even more

teslanick17:02:09

I feel like you’re skipping over a lot of intermediate context 😉

ghadi17:02:19

but caching is the big difference

teslanick17:02:04

What do you mean by caching here?

ghadi17:02:41

when you realize a seq, it only happens once

ghadi17:02:56

when you realize an eduction, it happens every time

ghadi17:02:03

it being side-effects + the xforms

teslanick17:02:50

Aha! That is an important difference

teslanick17:02:56

I’m trying to get a handle on transducers by reimplementing them in a more familiar environment (JS) using a relatively familiar-to-me primitive (generators). Thus far it is going adequately well. The explainer on the Clojure website is kinda hard to grok in theory, but messing around in the repl makes it somewhat more obvious what’s going on, poking at a reimplementation makes it easier to see the holes in my understanding

gibb18:02:48

I want to create a plot with a value distribution as follows: From week 0 to week 18, distribute the necessary goal weight for each week to hit a certain end goal by week 18. This is going to be used for a fun graph and bet with a coworker who is also trying to loose some weight. Basically I want to have weigh-ins every week and a small penalty if you're above the goal weight for that week

gibb18:02:22

This function gets me pretty close:

(defn weigh-in [starting-weight]
  (->> (range weeks-remaining)
       (map #(hash-map :week % :goal-weight (- starting-weight (* 0.8 %))))))

gibb18:02:56

Bah of course it's 1st grade math problem when I type it out 😄

gibb18:02:58

Nevermind...

Baris Aydek18:02:56

is there any easy way to parse/read keywords in a namespace? So I can iterate over them? Otherwise I need to copy every used keyword in that file/namespace and put in a list manually, which I don't want to do

andy.fingerhut18:02:29

There are ways, but the difficulty of doing it correctly depends upon how general you want to be across all legal Clojure code constructs, or just one file that has a limited subset of Clojure features used (and which ones those are).

andy.fingerhut18:02:07

e.g. the clj-kondo and Eastwood linters can read most Clojure source code, as can clojure.core/read , with the potential down side that clojure.core/read can have side effects of executing arbitrary code. That is not a problem if it is your own code, and you know it doesn't do dangerous things when you read it: https://clojuredocs.org/clojure.core/read

Baris Aydek18:02:19

oh, there is no dangerous stuff going on actually

andy.fingerhut18:02:21

It can be made more challenging if the code uses keywords like ::foo or aliases, e.g. :my-alias/bar -- if your code doesn't use those, then you don't have to worry about that.

andy.fingerhut18:02:16

I bring it up to warn you. If you are doing this for your own personal code, no worries. If you want to publish a tool to do this and run it on arbitrary Clojure source code, you might want to be safe, or at least have your documentation warn people of the dangers.

Baris Aydek18:02:17

ok i'll be careful. thanks

andy.fingerhut18:02:58

If your file doesn't use ::foo nor :my-alias/bar , and you want to be open to potential arbitrary code execution, clojure.core/read repeatedly run on a file should get all of the source code forms as Clojure data, e.g. nested lists, vectors, maps, symbols, keywords, numbers, etc.

andy.fingerhut18:02:16

If your code does use ::foo and/or :my-alias/bar then clojure.core/read should work after evaluating the appropriate ns forms, e.g. the one at the top of your source code file.

andy.fingerhut18:02:37

they will be expanded into their fully qualified versions

Baris Aydek19:02:00

what about something like ns-map ? this one returns everything in java. So if it is on runtime, it can't be clojure code anymore?

Baris Aydek19:02:37

Actually in my case I use these keywords as an enum and string representations. Only in one def I have

(def products {::car "Car" ::ps4 "PS4"})

andy.fingerhut19:02:47

The return value from ns-map doess not contain the full code of the namespace, and thus not the keywords in your code.

Baris Aydek19:02:12

What I really want is to get these ::car and ::ps4 and iterate over them

Baris Aydek19:02:49

because in the future I will add another product to that map

andy.fingerhut19:02:11

Clojure does not store the forms read when it loaded your code anywhere.

Baris Aydek19:02:00

Actually I just realised I can simply get the keys of that map

andy.fingerhut19:02:05

The data structures it reads are temporarily in memory while the Clojure compiler compiles it to JVM byte code (or JavaScript in the case of ClojureScript), but then that data is discarded.

andy.fingerhut19:02:53

Sure, if there are particular data structures of your code that have the keywords you care about, then just walk the contents of those data structures.

andy.fingerhut19:02:27

My answers were assuming that there were keywords throughout your code, and you didn't want to miss any of them, but that is a more difficult problem to solve correctly.

Baris Aydek19:02:27

true, you don't have to define them in one place, you can put them wherever you want

andy.fingerhut19:02:38

(keys my-map) returns a sequence of all keys in the map my-map . Keys of maps can be any Clojure value, e.g. numbers, lists, strings, keywords. The function keyword? returns true exactly when its argument is a keyword, vs. something else.

andy.fingerhut19:02:16

I don't know exactly what problem you are trying to solve, and how general you want to be about it.

Baris Aydek19:02:15

keys actually worked for me

andy.fingerhut19:02:07

OK. Sorry for leading you down the garden path, but I was taking you at your word that you wanted the generality of finding all keywords in a namespace. Finding all keys in a single map at run time -- much easier.

Baris Aydek19:02:30

No it's my fault actually. I was searching how to parse/read it since yesterday for no reason

Baris Aydek19:02:52

But thank you anyway it helped a lot to understand what's going on behind the scenes. sorry for the misunderstanding though

andy.fingerhut19:02:41

Clojure has quite a few functions built in for manipulating maps, vectors, lists, sets, etc. The Clojure cheatsheet tries to categorize what kind of functions are useful for what kinds of data structures, and may be helpful: https://clojure.org/api/cheatsheet

andy.fingerhut19:02:24

Clicking on the "source repo" link on that page gets you to other variants of it, e.g. ones that let you narrow down the list of results by typing a string, and showing you the doc strings just by hovering the cursor over the function name.

Baris Aydek19:02:16

this is nice!

Baris Aydek19:02:01

I put it on my bookmarks

Endre Bakken Stovner19:02:50

I cannot find any vars in my namespace even though they are there. I have stuff like

(def main-workflow-file (atom []))

(def five 5)
but when I do (dir my.namespace) I just get nil. What am I doing wrong?

noisesmith19:02:14

how are you loading your namespace?