Fork me on GitHub
#beginners
<
2019-05-07
>
k.i.o01:05:28

calling typed language untyped is incorrect, right?

k.i.o01:05:41

Some people keep calling dynamically typed languages as "untyped" which sounds wrong to my russian ear because to my understanding untyped == typeless.

seancorfield01:05:17

@k.i.o Right. Clojure is not untyped. It is fairly strongly typed. It is just dynamically typed -- so type checks occur at runtime, not compile-time.

seancorfield01:05:45

(hence the various ClassCastException errors you can get from incorrectly Clojure code)

seancorfield01:05:42

And Clojure does also have the ability to define new types (`deftype`, defrecord, defprotocol, etc) that are actual Java (class) types.

iagwanderson03:05:04

Hi ppl, what are you using to create Fake data? I found a library called Faker but it seems rather incomplete when compared to other languages such as Python

seancorfield04:05:07

@iagwanderson Can you be a bit more specific about what you're looking for? clojure.spec allows for some pretty impressive "fake data" creation.

iagwanderson04:05:54

I have some schemas on graphql that I want to populate with fake data (company names, full names, dates, numbers, addresses) because some of my providers do not have staging environments and I want to provide this feature to me direct users

tabidots04:05:46

I just discovered that you can use into with a transducer. I’m looking at some code written by someone else that uses the form (into [] f coll). How is this different than (mapv f coll)? Are there performance gains (especially if you are into-ing into a {})? Just wondering if there is anything worth refactoring in code that I’ve written.

seancorfield05:05:06

They are pretty much identical, based on their source @tabidots

seancorfield05:05:23

You mention (into {} ...) and that is different to (into [] ...)

seancorfield05:05:30

I find (into [] (map f) coll) a little clearer in intent than (mapv f coll) -- since the former makes it clear you are applying a transform to coll and "pouring the result into a vector".

seancorfield05:05:06

Actually, (into [] (map f) coll) and (mapv f coll) are the "pretty much identical" forms.

tabidots05:05:34

Yeah, with {} I was curious if there are any performance gains to be had. I’m just so used to doing map-`reduce`

seancorfield05:05:00

There's reduce-kv for reducing a map.

seancorfield05:05:42

Where the transducer form starts to shine is when you are combining multiple map/`filter` operations. Then it's definitely both clearer and faster.

tabidots05:05:28

Gotcha. Then you use eduction and put all the map-`filters` in the eduction right?

seancorfield05:05:40

I don't use eduction at all.

tabidots05:05:06

comp, then?

seancorfield05:05:23

(into [] (comp (map f) (filter p) (map g)) coll)

tabidots05:05:52

gotcha. So my first instinct would be to write

(->> coll
     (map g)
     (filter p)
     (map f)
     vec)

seancorfield05:05:12

That creates intermediate collections for each piece of the pipeline.

seancorfield05:05:20

The transducer version does not.

tabidots05:05:11

ahh. Good to know. I’ll have to hunt for some places where I can apply this. But in the case of just one transformation, there would not be a major difference (other than how the code “reads”) doing it with into res xf coll vs. a single map or reduce-kv, right? (Assuming a list and a vector are equally fine)

seancorfield05:05:28

map is lazy, mapv is not.

seancorfield05:05:34

mapv uses a transient vector so it's efficient like into []. reduce-kv relies on the collection supporting kv-reduce via a protocol. I'd have to dig into the source to see how that plays out.

tabidots05:05:40

oh! Didn’t know that either. Lazy sequences seem to be a bit of a bottleneck in some of the things I’m doing, or at least they’re taking a lot of horizontal space on the async-profiler flamegraphs.

seancorfield05:05:03

If you don't need laziness, avoid it if you care about performance.

tabidots05:05:01

Yeah I have one keep-indexed function that is really slowing things down—I tried using map with the coll as c1 and some range as c2 and it was even slower. But I didn’t try mapv

seancorfield05:05:01

There are situations where laziness is very important and you need it. Most situations neither need it nor care enough about performance to avoid it.

seancorfield05:05:46

mapv with multiple collections won't be any faster than map

seancorfield05:05:03

Look at the source. mapv is efficient for a single collection.

seancorfield05:05:57

Interestingly, keep-indexed has a transducer arity that will likely be more efficient for you.

tabidots05:05:17

Thanks! I will give that a go

seancorfield05:05:33

user=> (quick-bench (keep-indexed (fn [x y] x) (range 100)))
Evaluation count : 17056068 in 6 samples of 2842678 calls.
             Execution time mean : 35.779785 ns
    Execution time std-deviation : 4.136766 ns
   Execution time lower quantile : 29.745389 ns ( 2.5%)
   Execution time upper quantile : 39.122814 ns (97.5%)
                   Overhead used : 1.639129 ns
nil
user=> (quick-bench (into [] (keep-indexed (fn [x y] x)) (range 100)))
Evaluation count : 229200 in 6 samples of 38200 calls.
             Execution time mean : 2.739593 µs
    Execution time std-deviation : 134.067772 ns
   Execution time lower quantile : 2.606451 µs ( 2.5%)
   Execution time upper quantile : 2.893934 µs (97.5%)
                   Overhead used : 1.639129 ns
nil
Big difference in speed.

seancorfield05:05:14

The wrong way in this case 🙂

seancorfield05:05:27

So it isn't faster.

seancorfield05:05:36

I sort of assumed it would be...

seancorfield05:05:33

Criterium is your friend here.

seancorfield05:05:20

Oh, I wonder if my benchmark check isn't realizing the lazy sequence... just a sec...

seancorfield05:05:56

Hahaha... yeah, just shoot me! Lazy benchmarks are really fast when they don't do anything!

user=> (quick-bench (into [] (keep-indexed (fn [x y] y)) (range 100 200)))
Evaluation count : 232302 in 6 samples of 38717 calls.
             Execution time mean : 2.605172 µs
    Execution time std-deviation : 61.792867 ns
   Execution time lower quantile : 2.533126 µs ( 2.5%)
   Execution time upper quantile : 2.667096 µs (97.5%)
                   Overhead used : 1.639129 ns
nil
user=> (quick-bench (doall (keep-indexed (fn [x y] y) (range 100 200))))
Evaluation count : 149862 in 6 samples of 24977 calls.
             Execution time mean : 4.093292 µs
    Execution time std-deviation : 40.998696 ns
   Execution time lower quantile : 4.052769 µs ( 2.5%)
   Execution time upper quantile : 4.145466 µs (97.5%)
                   Overhead used : 1.639129 ns
nil

seancorfield05:05:46

So, yes, the transducer version of keep-indexed is faster... which at least matches my expectations! @tabidots

seancorfield05:05:46

At least this way you can combine the transducer form of keep-indexed with other things to avoid creating intermediate collections.

tabidots05:05:00

Wow, thanks so much for this tip! It just shaved ~9.5s off of the running time for this algorithm I’m working on, given a certain very large test value 😮 went from 41s to 38s yesterday and now 29s. Awesome! And now I see the power of into [] xf coll 😁

seancorfield05:05:16

Clojure performance can be a bit mysterious at times but avoiding laziness and avoiding boxed/checked math can really help.

seancorfield05:05:40

Sometimes you just have to break out the ol' loop/`recur` tho'...

tabidots05:05:59

hehe yeah I’m currently on a campaign to eliminate all the loop/`recur`s where I can 😆

iagwanderson11:05:08

hi, is it possible to execute a function without having it required in the namespace? I thought fullyqualified names would do that, but it's not working

iagwanderson11:05:23

my project name is clj-integrator and I have a function called first-name on the namespace faker so I placed it on a edn file like {:Name clj-integrator.faker/first-name and I expected to read the edn file and call the function on another namespace

alexmiller12:05:30

no, you need to cause the namespace to be loaded somehow

alexmiller12:05:17

one helpful function for that (as of Clojure 1.10) is requiring-resolve, which takes a symbol, requires the namespace, resolves the symbol to a var and returns it

alexmiller12:05:44

so you can do something like ((requiring-resolve 'faker/first-name)) - the inner parens will resolve the function first-name in the namespace faker. The outer parens invoke the returned var, which will invoke the referenced function. You could place arguments in the outer parens as well if needed.

alexmiller12:05:42

note that project/artifact name is not part of your namespace unless you explicitly create a directory segment and namespace name to include it. it was unclear to me whether you had done that so I did not include it

iagwanderson12:05:36

yes, I have done that. But that's exactly what I was looking for. Didn't know about requiring-resolve

iagwanderson13:05:57

just worked 😃 Thanks

scott.archer14:05:22

Hi what does an @ before a threaded function do?

alexmiller14:05:33

@ is syntax for the deref function

alexmiller14:05:03

@foo == (deref foo)

scott.archer14:05:28

Thanks I’ll read up on that. Searching for @ was not helpful 😀

alexmiller14:05:37

there's a guide for characters like that

scott.archer14:05:51

So I’ve used that with atoms, but in this case I’m assuming it’s forcing computation. I think the aleph http client is returning a future. Does that sound correct, or is there something lazy going on with threading?

alexmiller14:05:47

sounds correct, deref works with futures too

alexmiller14:05:59

will block until future returns

alexmiller14:05:12

the threading is incidental here

scott.archer14:05:30

That makes sense thanks! Just had to understand it in that context.

nicholas.jaunsen14:05:57

I'm trying to generate data from this spec I made, but I'm getting an error Unable to construct gen at: [:type] for: :type

alexmiller15:05:58

::type is an opaque function to spec so it doesn't how to gen. here, the easiest fix is to change that spec to a set #{"NetworkGraph"}

alexmiller15:05:27

any time you have an enumerated value, that's probably better as it will gen automatically

alexmiller15:05:56

some other tips... ::version could be (s/nilable string?) ::metric could be (s/nilable string?) ::properties could just be map? (also it's defined twice) ::cost could be (s/or double? int?) - will gen, etc (but will conform differently so that may not be what you want)

scott.archer15:05:10

I have a question about making an http post request using aleph's client. I can't seem to pass in authorization headers.

scott.archer15:05:38

When I try to pass in headers as a map, I get an IllegalArgumentException about how only ' ' and '\t' are allowed after '\n'

scott.archer15:05:03

Is this most likely a bug in aleph or netty, or am I passing the wrong thing into "headers"? I assumed "headers" would just be a map, so I'm not sure how I can mess that up.

haus15:05:10

Can you show how you’re trying to call it now?

scott.archer15:05:34

(defn make-auth-request! [url headers] (d/chain (http/post url {:body "grant_type=client_credentials" :headers headers}) :body))

scott.archer15:05:54

I'm just passing in a map with a single "Authorization" header for basic authentication.

scott.archer15:05:52

Honestly it's probably me not understanding the documentation well enough, but it seems like it would be straight forward.

scott.archer15:05:14

I'm just trying to make an authenticated http request. I've done this with clj/http without issue.

scott.archer17:05:29

I have another simple question. I have a list that I'd like to convert into a map. I'm struggling to find an example. (:k1 "v1" :k2 "v2" :k3 "v3") to {:k1 "v1", :k2, "v2", :k3 "v3"}

noisesmith17:05:26

(apply hash-map l)

ymussi17:05:02

How can i resolve this problem ?

ymussi17:05:03

Exception in thread “main” Syntax error compiling deftype* at (flatland/ordered/set.clj:19:1).

noisesmith17:05:30

my first guess is an old version of flatland which a more recent clojure compiler doesn't like

iagwanderson17:05:40

Hi, I am using the lacinia and lacinia-pedestal library and I want to deploy my project. All my schemas definitions are under a folder at resources/schemas/ however, after I built the jar file using lein uberjar I tried to run the jar file and I got the following error Caused by: java.lang.IllegalArgumentException: Not a file: jar:file:/graphql/integrador.jar!/schemas It seems that I need to include something on my project.clj to include the schemas. (ps: my project is called integrador)

noisesmith17:05:26

@iagwanderson resources inside a jar can't be accessed with the File API, if you are using it, you can replace it with the resource API

noisesmith17:05:27

usually this is a question of replacing (java.io.File. foo) or ( foo) with ( foo) - nothing else about the code needs to change

iagwanderson17:05:36

I created the following function to list all schema files on the folder:

(defn list-files-resource
  "Function to list all files inside a `folder-name` that
  is present in resources/."
  [folder-name]
  (-> (io/resource folder-name)
          io/file
          file-seq
          rest))

noisesmith17:05:08

that's trickier - you need to find all resources that are children of that path

noisesmith17:05:24

file-seq and io/file are not usable without unpacking the jar into the file system

donaldball17:05:09

You may just need to add "resources" to your :resource-paths vector in project.clj

noisesmith17:05:31

@donaldball that doesn't help - the file api doesn't work on things inside jars

donaldball17:05:11

I speculate that his uberjar simply isn’t including the schema resources

noisesmith17:05:23

no, the file api never works on things that are inside jars

noisesmith17:05:26

this is a common problem

deep-symmetry23:05:11

One thing that I ended up finding tremendously useful in adding a big new feature to one of my open-source projects this winter was learning about the FileSystems API introduced in Java 7 (https://docs.oracle.com/javase/8/docs/api/java/nio/file/FileSystems.html). Using that via Clojure Java interop allowed me to effectively mount a Jar file as if it was a file system, and read and write file entries inside the Jar file, so my users only had to deal with a single file, but I could organize complex text and binary values inside it for them.

noisesmith23:05:28

interesting - a lot more than what we usually need, but it's really cool to know how to do it

deep-symmetry00:05:39

Yes, this was definitely the only time I have needed to do something like this, and I don’t expect to again soon! I also suspect it is a rather lightly used part of the Java class library, because I found some bugs in it that I had to work around. 😆

iagwanderson17:05:44

didn't know about that. I'll check the sw answer

noisesmith17:05:51

resources is on the resource-paths config by default, you need to go out of your way to make that not work

noisesmith17:05:46

@donaldball while it's possible the resource isn't inside that jar, using io/file guarantees not finding them, even if they are there

iagwanderson17:05:09

thanks for the help guys

scott.archer21:05:13

is there an easy way to "clojurify" keys in a map?

scott.archer21:05:34

I'm getting some keys back from an API like {"access_key" "12345"}

scott.archer21:05:00

I wrote several functions to manually convert it, but was curious if there was an existing way. Right now I'm converting _ to - prepending : and then converting that to a symbol.

noisesmith21:05:15

there's a project camel-snake-kebab which does that kind of transform

scott.archer21:05:34

I have it working, but what I'm doing doesn't seem great.

scott.archer21:05:54

ended up using rename keys and interleaving the original keys with the new symbols.

noisesmith21:05:23

sometimes the transforming is just introducing complexity - don't discount the utility of just using the original strings instead of re-casing and using keywords

scott.archer21:05:24

I could use them as they are, I'm still trying to figure out what best practice is for these things.

scott.archer21:05:54

It was definitely beneficial for me to write the code to convert, but I may delete it now. I learned how to do it at least.

noisesmith21:05:54

I like to distinguish readability vs. beauty - improvements that actually improve readability are good, if they just improve beauty without making things more readable than they were before, it's probably not a net gain

scott.archer21:05:36

It's definitely harder to read through the code that does the conversion.

scott.archer21:05:43

So probably better without it.

atamiser21:05:52

rename-keys also isn't the best way to bulk-rename keys: (->> m (map (fn [[k v]] [(keyword k) v])) (into {})) might be better

atamiser21:05:33

I wouldn't discount the value of keyword accessors in terms of redability: I prefer (:access_key map) to (get map "access_key")