Fork me on GitHub
#beginners
<
2017-11-09
>
oliv02:11:35

hey guys, is there any way to do something like that

(defn foo [] "foo-fn")
(defn bar [] "bar-fn")
(def fns ['foo 'bar])

((first fns))

oliv02:11:00

I would love to inoke the function on ((first fns))

oliv02:11:09

is is possible ?

oliv02:11:23

((resolve (first fns)))

oliv02:11:34

problem solved guys ^^ ! thanks

seancorfield02:11:13

You can just use the functions directly @oliv

(def fns [foo bar])
((first fns))

seancorfield02:11:01

(if you want to have the vector of fns refer to updated versions of those functions, i.e., if you re-`defn` foo or bar, then you can use Vars:

(def fns [#'foo #'bar])
((first fns)) ; "foo-fn"
(defn foo [] "new foo!")
((first fns)) ; "new foo!" now

oliv03:11:21

thanks @seancorfield

noisesmith03:11:26

if you had a list of symbols because they came from user input, you can use a hash-map from string to function, which ensures you don't expose an arbitrary code execution

oliv03:11:57

good idea ^^ { “foo” foo }

rabmcfergus13:11:43

Hi all. I've been making my way through Clojure For The Brave and True, and have a question about Chapter 5: https://www.braveclojure.com/functional-programming/

rabmcfergus13:11:53

The example for the chapter is the Peg Thing game, which is a puzzle board that involves moving pegs from position to position and removing pegs that are leapfrogged. The user interaction is done in a functional style, where the board is passed from function to function in a recursive style. For example the function:

rabmcfergus13:11:32

The function prompt-move prints a prompt to the terminal, gets the user input, and if the move is valid then in turn calls the function user-entered-valid-move with the board as the argument. My question is: does this style of programming not pose a problem in that it keeps adding to the stack? In a small game like Peg Thing the stack could never grow very large, but in a much larger game, could this not result in stack overflow?

noisesmith13:11:30

there are tools like recur, reduce, iterate, and trampoline that avoid growing the stack

rabmcfergus13:11:31

Thanks @noisesmith, I've read about some of those and I'm looking forward to learning them in due course (noob here!). Am I right in thinking that the example given does grow the stack in a way that could be dangerous in a larger game?

noisesmith13:11:56

yes - it's likely to avoid muddying things by introducing too many concepts at once

noisesmith13:11:56

I mean, that could use trampoline and be better behaved, but that changes it from a intermediate beginner lesson to an introductory advanced lesson or something

chris13:11:08

in many lisps this problem is mitigated by having automatic tail call optimization

chris13:11:39

the jvm does not support automatic TCO (and it can be quite dangerous if you screw it up)

noisesmith13:11:43

but java doesn't support goto across method boundaries so we can't have that

noisesmith13:11:01

we could have automatic self-call TCO and it was intentionally not implemented

chris13:11:13

so, clojure supports this construct through recur in order to be explicit

noisesmith13:11:44

right - what I am saying is that's not because of the JVM, that's because Rich noticed that a common source of bugs was programmers assuming something was a tail call when it was not

chris13:11:23

oh, I thought it literally wasn't possible in the JVM as it stood now, but I agree that making it explicit (and an error if done incorrectly) is a better way to do it

rabmcfergus13:11:26

@noisesmith @chris OK, that makes sense. In general, is this style (continuation passing style?) with use of trampolines etc typical for ui design in Clojure - apart from stack blowing it looks to be quite elegant - or are there other/better ways

noisesmith13:11:27

the compiler could trivially detect and optimize self tail calls if we chose to

noisesmith13:11:31

it's just a loop!

sundarj14:11:02

yeah, iirc he was afraid that implementing it automatically for self-calls would trick people into thinking there was full TCO

noisesmith14:11:28

@rabmcfergus yeah - trampoline generalizes it by requiring you to pass an actual continuation (as a function of no args) for the next step

noisesmith14:11:20

and yes, it's idiomatic (though using loop is totally valid too for example - depends on how the thing is structured)

noisesmith14:11:46

sometimes you have enough distinct cases that having a separate function for each is clearer, sometimes having a straightforward loop body is clearer

rabmcfergus14:11:40

Idiomatic was the word I was looking for 😄, thanks for clarifying it

noisesmith14:11:59

another fun thing about trampoline (besides the fun sounding name that is) is that it's one of the few cases where letfn is quite useful - since the functions defined by letfn can be mutually recursive

sundarj14:11:46

@rabmcfergus @chris here is Rich's fuller explanation of the lack of tail calls in clojure: https://gist.github.com/reborg/dc8b0c96c397a56668905e2767fd697f#why-no-tail-call-optimization

rabmcfergus14:11:11

Thanks guys, that's given me something more to look into :+1:

tjtolton16:11:53

Okay, so I'm thinking about taking another swing at clojurescript. Some of our backend servers are using Node.js and I'd really like to try using clojurescript to run another rest endpoint on the same server. Anyone know how I'd go about doing that? I know there are hybrid clojure/java projects.. are there hybrid node/cljs projects too?

tjtolton17:11:19

niiice. excellent, thanks

tjtolton17:11:42

I'd heard of lumo, but cljs moves pretty fast, I was hoping it hadnt become obsolete

lovuikeng17:11:58

while you're at it, @tjtolton you might find this dynamic admin tool handy https://github.com/juxt/mach

tjtolton17:11:47

cool, that's some healthy documentation! I'll look into this 🙂

erp1217:11:31

Is there a way to define a type or record and a behavior for when it is passed as the map argument to assoc?

noisesmith18:11:17

@erp12 yup - you can make a deftype that implements Associative, which has three methods containsKey, entryAt and assoc https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Associative.java

noisesmith18:11:10

you’ll probably also want to implement some of the other superclasses of hash-map or vector, depending on what kind of thing it’s supposed to be? but the Associative methods could be enough

noisesmith18:11:11

@erp12

(ins)user=> (deftype Foo [m] clojure.lang.Associative (containsKey [_ k] (contains? m k)) (entryAt [_ k] (find m k)) (assoc [_ k v] (println "ASSOC!") (->Foo (assoc m k v))))
user.Foo
(ins)user=> (def f (Foo. {:a 0}))
#'user/f
(ins)user=> (.entryAt f :a)
[:a 0]
(ins)user=> (def g (assoc f :b 1))
ASSOC!
#'user/g
(ins)user=> (.entryAt g :b)
[:b 1]
(ins)user=> (.entryAt g :a)
[:a 0]
user=>
this is partial - it errors on get for example, but it’s a start and it should be straightforward how to work forward from there

noisesmith18:11:00

@erp12 and for reference, I figured out how to do that by looking at (source assoc) and seeing what methods in the java impl that leads to

noisesmith18:11:36

this info might also be useful

(ins)user=> (supers (class {}))
#{java.util.concurrent.Callable clojure.lang.IMapIterable java.io.Serializable clojure.lang.IHashEq clojure.lang.Counted clojure.lang.IPersistentCollection java.lang.Runnable java.lang.Object clojure.lang.APersistentMap clojure.lang.IPersistentMap clojure.lang.IObj clojure.lang.ILookup clojure.lang.MapEquivalence clojure.lang.Seqable java.util.Map clojure.lang.IEditableCollection clojure.lang.IKVReduce clojure.lang.AFn clojure.lang.IMeta clojure.lang.Associative java.lang.Iterable clojure.lang.IFn}

tkjone18:11:17

I have a vector like this:

(def events [ [ 1 2 ] [ 4 5 ] ])
I would like to create a new vector from the above by comparing the first item to the second, doing some logic and than potentially returning one of the items, or none. What would be the best way to do this? My thought was to use reduce, but the best I came up with was
(reduce #( ...logic here) [0 [] ] events)
The part that reads [0 []] the 0 is meant to record the current index, so I can find the next element and the [] part is where I would store the items I potentially returned if they meet my functions logic

noisesmith18:11:32

if at each step you want to compare the current input and the next input, you might want (reduce f [] (map vector (cons nil events) events))

noisesmith18:11:44

so each input is a two element vector [prev-event this-event]

noisesmith18:11:16

(map vector (cons nil events) events) is almost like (partition 2 1 events) but I’ve found many cases where the first one is more useful (I don’t know what a good name for it would be though)

tkjone19:11:59

There is quite a bit to unpack here.

tkjone19:11:03

I see what the (cons nil events) line will output:

nil [1 2] [4 5]
But I am curious why we are joining nil to it?

noisesmith19:11:34

because at step one, there is no previous

noisesmith19:11:52

the idea being, at each step, you see what the previous was, and what this one is

noisesmith19:11:26

if nil is a valid sequence element, you can use another token, like ::missing - as long as you can reliably test for it and act on it accordingly it doesn’t matter so much what it is

tkjone19:11:39

Awesome. Okay, I see that now.

tkjone19:11:06

Given that the first one never has a previous, would it make more sense to reverse those two? For example, given my vector is [ [ 1 2 ] [ 4 5 ] ] and when I reduce over it the first item I will get is [1 2] I want to compare [1 2] against the next item, which is [ 4 5 ]

tkjone19:11:59

so would I want a list more where the :missing is at the end, not the beginning so I know there is nothing left to compare to and handle that case appropriatley?

tkjone19:11:21

My thought is that somewhere in the reduce function logic I would need to have a condition to check if the other item in my pair of current and prev items is :missing and when this is the case, don't worry about comparing

noisesmith19:11:11

yeah - if you use missing instead of :missing it’s namespaced to your specific ns so much less likely to clash (not that it’s a likely clash, but this is a normal usage of namespaced keywords)

tkjone19:11:33

That makes sense. So this is generally the way this is handled in functional programming?

noisesmith19:11:50

it’s a common pattern, yeah - multi-value accumulators are also totally fine, but it can simplify your reducing logic if you can pre-process the input to give you the two items you are interested in at each step

tkjone19:11:19

awesome. yes, that was my concern - which one was better practice and whether or not multi-value accumulators were a good practice. I imagine its just right to for the job deal.

noisesmith19:11:53

yeah - it’s a question of reducing complexity by isolating concerns, and if the logic of determining which input each step needs is self contained, it’s a win to do that separately as a pre-processing step instead of mixing it with your per-item processing logic

tkjone19:11:40

Hypothetical; because we are processing the list twice, how does this affect our space time complexity? Or does Clojure have a sexy way of handling this so everything is still fast?The reason I ask if because if I have to process once, and lets say there is a another processing required before I start to reduce, does this add to the time complexity negatively?

noisesmith19:11:17

“processing the list twice” in this case means two args in a vector instead of one arg, and two elements in heap instead of one if the input was lazy

noisesmith19:11:34

it still doesn’t hold on to previous items as long as you manage the lazy input smartly

noisesmith19:11:39

“processing the list twice” also means calling the method that asks for the next item in a list twice as many times, but if that is a bottleneck, you probably shouldn’t be using clojure in the firstplace and you probably should not be using the jvm 😄

tkjone19:11:06

haha I feel I missed some of the nuance in your last comment, under which condition would you not be using Cojure/JVM?

noisesmith19:11:23

if asking for next on a list twice as many times makes your app too slow

noisesmith19:11:37

what I mean is, when clojure accesses the rest of a sequence, the work was a) already done when the sequential item was created because it wasn’t lazy or b) done once because it was lazy but cached and not done a second time

noisesmith19:11:04

so the only extra work to access it twice is a pointer lookup, and if pointer lookups are making your app too slow, you probably want a different programming paradigm

noisesmith19:11:11

like using C or assembly

shaun-mahood19:11:24

Are there any good examples of a secure full stack Clojure/CLJS site using something like Auth0 or Azure for user identity? I can find a lot of examples and libraries for parts of it but I'm having trouble putting together the whole picture (lots of Java examples but I have experience with buliding Java sites).

stvnmllr220:11:47

@shaun-mahood Not public, but I'm building one and spent a while getting auth0 to work. (well, fb initially) But it relies on server redirects. I didn't know how to deal with js and oauth securely for sure. (I did try)

shaun-mahood20:11:06

If there's anything you can share on the server side that would be helpful - there's a few auth0 libraries but I haven't found one that really clicked. This is the most specific resource I've found on the cljs side so far http://randomlurker.eu/clojurescript/re-frame/2017/05/22/re-frame-auth0-authentication.html

stvnmllr220:11:59

yeah, mine isn't that secret. I can share the repo if you want

stvnmllr220:11:15

Just a proj i'm working on the side

stvnmllr220:11:36

on bitbucket

seancorfield20:11:56

@shaun-mahood We've done AzureAD integration -- but like @stvnmllr2 ours is internal only, sorry.

drewverlee20:11:07

Is there a good channel to ask more general software questions?

drewverlee20:11:38

I want to ask about when to throw exceptions, i’m not sure thats really clojure specific.

jaymartin21:11:01

@drewverlee I’m not sure about where to ask general questions, but exception handling in Clojure is different than in Java so feel free to ask here or in #clojure. In Clojure, you don’t have to deal with checked exceptions. You still have to clean up after using resources and respond to exceptions where they might occur in your code. Depending on who you ask, they might be a code smell if overused.

drewverlee21:11:27

Thanks jay. My question is very general. When is appropriate to throw an exception. My guideline is that its appropriate when the program cannot reasonable continue. 1/0 is a reasonable time to throw an exception on the core libs part because it cannot continue the rest of the program, as 1/0 is undefined behavior e.g what (1/0) + 5. However it wouldn’t make sense to build a calculator app, take the request 1/0 and throw an exception on validating the input. The request is reasonable, you can handle it by inspect the math function and noticing the error. OR you could catch the error when you evaled the callers math equation, but you wouldn’t re-raise it. I ask because i seem the later behavior quite often, catching an error then re-rasing it or a different one. I can’t think of a scenario where this makes sense, now i feel were using exceptions as flow control.

jaymartin14:11:13

@drewverlee I ran across a video which uses a higher order function, fnil, instead of a try-catch block. This is tangentially related to your question but I think the videos by @ are quite excellent and worth sharing. https://youtu.be/uDOvBAcApC4

drewverlee15:11:43

Cool I'll was take a look

noisesmith21:11:49

Exceptions are emphatically not for situations when a program cannot continue

noisesmith21:11:56

if a program cannot continue, throw an Error

noisesmith21:11:15

if somebody N layers up might recover and redo something, but maybe not your immediate caller, throw an Exception

drewverlee21:11:17

> if somebody N layers up might recover and redo something, but maybe not your immediate caller, throw an Exception I’m not sure how to understand this. if someone higher up the stack can handle the exception, then why throw it? If by layers you mean a program which needs to defer handling to another, then i think that makes sense. Like if i write a library, i might throw an exception given some inputs and pass that back to you because i’m not sure what makes sense in your use case and You have to handle it other wise the program cannot continue reasonable. (we can’t just pass you a msg saying things went bad, we need to blow up if you dont take an action).

noisesmith22:11:41

The point of Exceptions is to give code higher in the stack some chance to recover. That's why they exist. There are other Throwable objects that aren't meant to be caught and recovered.

jaymartin21:11:12

I just learned something : )

noisesmith21:11:48

to be clear, I might be using “program” differently - Error means things are bad enough that the VM should likely exit

noisesmith21:11:00

(with caveats for repl based flow of course)

gdeer8121:11:34

@jaymartin have you seen this talk? https://www.youtube.com/watch?v=zp0OEDcAro0 some good nuggets in there

jaymartin21:11:33

No, I haven’t. Can’t wait! Thank you!

noisesmith21:11:41

but even with a repl, some Errors should just mean exit the vm - or otherwise you’ll risk corrupting resources outside your program

gdeer8121:11:32

I also think Stu's talk about ETL has the hottest take on Exceptions and Errors

gdeer8121:11:02

Yes, specifically the Hall and Oats reference

lvbarbosa21:11:24

does core.async set up and uses its own thread pool in Clojure?

lvbarbosa21:11:53

or is it sharing the same thread pools used by agents/futures?

lvbarbosa21:11:00

like send / send-off

jaymartin21:11:02

@lvbarbosa I assume that because core.async is its own library it will create and manage its own threads (via thread) or threadpool (via go). Maybe someone can confirm this assumption.

noisesmith22:11:54

thread uses an auto-expanding pool owned by core.async, go uses a much smaller fixed size pool, also owned by core.async

lvbarbosa22:11:04

got it, thanks guys!

jay.beldon22:11:16

help me? I’ve got a hash map returned to me, and I need to pick the key that closely matches this: Metadata/file*.xml : the * is a wildcard. The value of this key is my location of my file on a local file system. what’s the easiest and most straight forward way to wildcard this key and get at its value? Thanks.

dpsutton22:11:04

do you know the path on your local system?

jay.beldon22:11:11

yes, they value of the key is the full path to where the file is stored

dpsutton22:11:31

assuming that's in some var binding, the keyword would just be (keyword "Metadata" (str "file" path ".xml"))

jay.beldon22:11:33

so I have many keys/values since its a hashmap… and I need to wildcard one of the keys so it’ll match and return the value to me… does get-in work?

dpsutton22:11:40

no i'm not sure i understand then. wild card doesn't mean anything to me. i thought the situation was there are many keys in there with filepaths as part of the key. so you need to figure out which key is yours, and its the one that has the filepath to your machine

dpsutton22:11:42

you want the key that most resembles "Metadata/file [random stuff].xml"?

jay.beldon23:11:24

yes, so I have “metadata/file1.xml” “path/to/file1.xml”, “metadata/file2.xml” “path/to/file2.xml”, etc…}

dpsutton23:11:03

so you need to look in a directory and find out which xml file you have? and from that construct the key?

jay.beldon23:11:53

well, i think I have the keys… they’re in the map.

noisesmith23:11:18

(keys m) gives just the keys as a sequence

noisesmith23:11:37

then you can filter to select the one that matches something on your filesystem?

dpsutton23:11:16

so you have a map with a bunch of keys. and you know which file you have?

seancorfield23:11:53

(into {} (filter (comp (partial re-find #"Metadata/file.*\.xml") first) map-of-files) to filter hash map by wildcard on keys maybe?

noisesmith23:11:39

I'd use key instead of first, but yeah, that's pretty close to what I was thinking

seancorfield23:11:49

(into {} (filter (comp (partial re-find #"Metadata/file.*\.xml") first)
  {"Metadata/file1.xml" "path/to/file1" "stuff/file2.edn" "some/edn/file.edn" "Metadata/filexyz.xml" "path/to/filexyz.xml"}))
=>
{"Metadata/filexyz.xml" "path/to/filexyz.xml",
 "Metadata/file1.xml" "path/to/file1"}

jay.beldon23:11:20

ok, so if the filename that I cared about was called file_awesome.xml, then this would find it with using #“Metadata/file.*.xml”) first) map-of-files) ?

seancorfield23:11:47

or (filter (fn [[k _]] (re-find #"Metadata/file.*\.xml" k)) ...) if you don't like the comp version

noisesmith23:11:20

the reason I suggest key instead of first is because it does the same thing in the context we care about, and it's more specific - therefore clarifying intent slightly

seancorfield23:11:42

Yeah, assuming you have a map of filenames to paths -- both strings -- then that will filter for a key (filename) that matches whatever regex you provide, and produce a map containing just the matching files/paths.

dpsutton23:11:58

if you know the filename you care about, can't you just grab it? (get file-map "Metadata/file_awesome.xml"). That seems to be the way I'm reading it

seancorfield23:11:01

@noisesmith Agreed. I think the explicit (fn [[k _]] ...) is even clearer?

dpsutton23:11:21

yeah i always destructure [[k v]]

jay.beldon23:11:26

well, I won’t know the exact file name… I’ll know what it begins with or ends with, but not the whole thing…

seancorfield23:11:22

So you'll have (defn find-file [prefix extension map-of-files] ...) ?

jay.beldon23:11:04

ya, that looks right

seancorfield23:11:57

So you'll want something like (re-find (re-pattern (str prefix ".*\." extension)) k) (assuming extension would be, say, xml without the dot) -- since . is a regex special character...

seancorfield23:11:12

Hmm, maybe ".*\\." I think.

dpsutton23:11:56

and do you need to worry about ranking the results? if you can expect one and only one result to match you're golden. things get complicated if you need heuristics for a "best" match

jay.beldon23:11:26

I don’t think it’ll be ‘best’ match or anything that complicated

dpsutton23:11:54

can you tell me two of the keys in the map?

jay.beldon23:11:06

yep; 1 key would be “Metadata/zenith.xml” 2nd key: “Metadata/metadata.xml”

seancorfield23:11:15

(defn find-files [prefix extension map-of-files]
  (into {} (filter (fn [[k _]] (re-find (re-pattern (str prefix ".*\\." extension)) k)) map-of-files)))
(find-files "Metadata/file" "xml" {"Metadata/file1.xml" "path/to/file1" "stuff/file2.edn" "some/edn/file.edn" "Metadata/filexyz.xml" "path/to/filexyz.xml"})
@jay.beldon are your maps something like that?

jay.beldon23:11:56

yes @seancorfield

dpsutton23:11:18

and how do you know which file you care about?

seancorfield23:11:33

Sounds like you're looking for any files that match?

jay.beldon23:11:44

yep, so I think that I’ll know what it starts with… some files will have weird names like M1Cxb1.xml, the next will be M1C1b1.xml, but they’ll start with “M1C”

jay.beldon23:11:30

.. that said, they’ll only be one file that starts with M1C in my map…

jay.beldon23:11:58

so I just needed a way to wildcard the entire file name/key so I can find the one that starts with M1C

dpsutton23:11:23

and it doesn't matter if you have M1CA.xml and the key is M1CZ.xml?

dpsutton23:11:02

the touch thing is you don't know how much prefix is required to get a unique result. I think I would reduce over the filename you care about, using that prefix function that sean made earlier until you get one and only one key back from the map

dpsutton23:11:34

for instance, in "abcdefghijk.xml", you don't know if you're looking for "a*.xml", "ab*.xml", etc. so keep looking with increasing prefixes until you have a single match

jay.beldon23:11:18

gotcha, ok that makes sense…

dpsutton23:11:14

that's what i'm thinking. since you don't know what an important prefix is, keep going until you get a unique prefix

jay.beldon23:11:08

thanks guys!

dpsutton23:11:08

this is linear in the map size, so you could select keys out of the original map to keep decreasing the original map to only common substrings if it takes a while

dpsutton23:11:42

ie, in the second half of the if statement [(select-keys m matching) substring]