Fork me on GitHub
#beginners
<
2020-04-16
>
alidlorenzo01:04:27

giving a shot at my first macro 😎 I’ve grown to like JS’s destructuring syntax so I’m writing a macro that gives me something similar. below i’m converting symbols into key-value pairs in a map. Looks like a single splice inside a map returns a compile error bc it assumes it’s an uneven pair. Is it OK to just add an extra splice like I did below?

(defmacro dmap
  "Constructs map of destrusctured symbols `syms` and an optional map `m`.
   e.g. (def x 1) (dmap [x] {:y 2}) -> {:x 1 :y 2}"
  ([syms] `(dmap ~syms {}))
  ([syms m]
   `(identity ~(merge `{[email protected](reduce (fn [acc k] (conj acc (keyword k) k))
                                  []
                                  syms)
                        ;; extra splice to prevent "uneven pairs" compile error.
                        [email protected][]}
                     m)))

Cameron02:04:22

I dunno, that does seem a little hackey, I think in this sort of scenario I usually just see something like this

(into {} (reduce (fn [acc k] (conj acc [(keyword k) k]))
                 []
                 syms))

Cameron02:04:37

(also note that in this case, instead of (conj acc (keyword k) k) you'd have (conj acc *[*(keyword k) k*]*) , since that's easy to miss from a glance)

dpsutton02:04:40

the [email protected][] works around a quirk in the reader and I would problably do (merge (into {} ~(reduce`

dpsutton02:04:34

I like your macro and I've wanted one like it for a while

dpsutton02:04:30

and i bet your version will error on the following: (dmap [x x] {})

alidlorenzo02:04:39

that’s a good alternative, thanks @zdot101 and thanks @dpsutton *, disliked having to read lots of {:foo foo} declarations and gave me a good excuse to write a macro 🙂

dpsutton02:04:41

the into version will just overwrite

alidlorenzo02:04:38

hm that’s a good edge case, what would you except result to be? right now if x is 1 (dmap [x x] {}) returns {:x 1}

alidlorenzo02:04:14

oh wait that’s with the old version, haven’t rewritten it with into

dpsutton02:04:39

oh i expected the version that made a literal map to explode with duplicate keys

dpsutton02:04:57

yeah i thought this would blow up {[email protected][:a :d :a :c] [email protected][]} but the fact that the [email protected][] is present kinda gives away the reason it doesn't blow up

alidlorenzo02:04:16

why did you think above would blow up?

alidlorenzo02:04:03

dmap version to is out 😜

(defmacro dmap
  "Constructs map of destrusctured symbols `syms` and an optional map `m`.
   e.g. (def x 1) (dmap [x] {:y 2}) -> {:x 1 :y 2}"
  ([syms] `(dmap ~syms {}))
  ([syms m]
   `(identity ~(merge (into {} (reduce (fn [acc k] (conj acc [(keyword k) k]))
                                       []
                                       syms))
                      m))))

alidlorenzo02:04:10

thanks again guys

dpsutton02:04:24

the reader does some checks on map literals as you've already seenwith the [email protected][] to ensure there were two elements. you can see duplicate keys check with

cljs.user=> {:x 1 :x 2}
Syntax error reading source at (<cljs repl>:1).
Map literal contains duplicate key: :x

👍 1
jeff tang03:04:21

#leiningen question: when I run lein repl, I have nREPL 0.6 , REPL-y 0.4.4, but nREPL should be updated to 0.7, no? I have lein v 2.9.3 and OpenJDK 11.0.2

seancorfield04:04:32

I think @U051BLM8F said that the next version of lein would bundle nREPL 0.7 since that's only just been released?

seancorfield04:04:30

Yeah, here https://metaredux.com/posts/2020/04/10/a-cornucopia-of-nrepl-updates.html under Misc "The next Leiningen release will bundle nREPL 0.7."

seancorfield04:04:12

@alidcastano You might also look at https://github.com/worldsingles/commons/blob/master/src/ws/clojure/extensions.clj#L148 for inspiration. It's based on a macro I saw @noisesmith post and also intended to return boilerplate when creating hash maps from local bindings.

👀 1
alidlorenzo04:04:37

interesting, that’s a much more complex utility. looks like it somehow accesses the let bindings and selectively includes them based on op keyword. will try and learn from it thanks

seancorfield04:04:49

There are usage examples in the comment below. LMK if you have any questions about it.

👍 1
noisesmith15:04:22

to be clear it's not just let bindings - it's all locals including loop, for, let, fn ...

noisesmith15:04:43

it even captures the weirdly named tmp names that destructuring creates but doesn' tusually expose

👀 1
seancorfield17:04:12

(and I carefully did not say "`let` bindings" in that local-map stuff but it would probably be clearer if I showed what the results are inline in the comment at the bottom of the file -- and added a destructuring example too!)

Uri10:04:39

Hi, thinking about using datomic/crux and wanted to ask some questions if anyone is knowledgeable about either. 1. Can I version a bunch of datoms, and then invalidate them all at once? 2. How would storing objects look like from javascript/python using http? 3. How would querying be done from javascript/python?

refset10:04:39

Hi @U011WV5VD0V - I work on Crux. A Crux document can definitely be thought of as equivalent to a "bunch of datoms" that get versioned and invalidated together. Storing and querying data over HTTP is supported: https://www.opencrux.com/docs#restapi HTTP only talks application/json right now though, so you would probably want to convert json<->edn yourself by shelling out to babashka or similar, until we implement official application/json support. Feel free to ask for help on #crux 🙂

Uri10:04:47

Thanks! Didn't know this channel exists, will ask there next time 🙂 So if I'm loading a bunch of entities like so [{age: 40, name: "john"}, ...] they get converted to datoms indexed etc' and I can tomorrow load a new batch and invalidate the previous one. Great. My other question is about setting crux up in a kubernetes cluster - is there a tutorial I can follow somewhere?

Uri10:04:54

you know what let me ask there for others' benefit

refset10:04:56

> So if I'm loading a bunch of entities like so > [{age: 40, name: "john"}, ...] they get converted to datoms indexed etc' > and I can tomorrow load a new batch and invalidate the previous one. > Great. That's pretty much it. A document is a version of an entity (so you also need to specify and ID), and the indexes are datom-like

introom12:04:41

is there a sorted multi set ?

andy.fingerhut13:04:28

There is no multi-set data structure built into Clojure, unless you want to think of a map from the set elements to integers that are the count of the number of times the element appears in the set as a kind of multi-set.

Mitch13:04:32

I would use a sorted map with the count as the vals

andy.fingerhut13:04:19

I would not be surprised if there is some 3rd party library somewhere for implementing a multi-set, but I don't recall a particular one off the top of my head.

Mitch13:04:31

Vectors give you everything that multisets do afaik

Mitch13:04:34

when would I want to use *command-line-args* rather than just parsing the args from my main function?

alexmiller13:04:53

there is no reason. command-line-args is useful if you are running as a script, not from a main function

alexmiller13:04:19

that is, clj foo.clj vs clj -m foo.clj

thanks2 2
jsn13:04:00

vectors don't give you e.g. fast membership check

👍 1
Mitch13:04:27

isn't it a hash tree?

Mitch13:04:31

ah thanks

alexmiller13:04:33

not hashed on value that is

alexmiller13:04:43

vectors give you fast lookup by index

Aviv Kotek14:04:12

i'm trying to build a co-occurence matrix of given documents and window-size, this matrix will represent the context (it n subsequent AND preceding tokens) of a token in the text. "Roses are red sky is blue" with windows size=2, would map roses: [are, red]. are: [Roses, red, sky]. etc... then this will go into a matrix. tokens will represent row/col in the matrix (term-to-term matrix). I have made it fairly easy https://stackoverflow.com/questions/35562789/how-do-i-calculate-a-word-word-co-occurrence-matrix-with-sklearn/61246990#61246990 with numpy and nested-loops updating the matrix, but somehow my https://github.com/akotek/vicarious/blob/master/src/vicarious/word2vec.clj seems to have big LOC, less readable and was more difficult to implement. I have used core.matrix so for the string example with n=1: (co-occurrence-matrix [["roses" "are" "red" "sky" "is" "blue"]] 1) => {:M [[0.0 0.0 0.0 1.0 1.0 0.0] [0.0 0.0 1.0 0.0 0.0 0.0] [0.0 1.0 0.0 0.0 0.0 1.0] [1.0 0.0 0.0 0.0 0.0 1.0] [1.0 0.0 0.0 0.0 0.0 0.0] [0.0 0.0 1.0 1.0 0.0 0.0]], :word2idx {"are" 0, "blue" 1, "is" 2, "red" 3, "roses" 4, "sky" 5}} are there better ways to iterate over the text and transform the data? is my usage of matrix operations correct? is there's any better practices for vectorized-programming with functional programming?

hindol15:04:04

I looked at the Python solution. You can achieve something similar with map-indexed.

(map-indexed vector ["roses" "are" "red" "sky" "is" "blue"])
;; => ([0 "roses"] [1 "are"] [2 "red"] [3 "sky"] [4 "is"] [5 "blue"])
Once you have the indices next to the words, you can have a similar algorithm as Python.

Aviv Kotek15:04:16

awesome. would there any way to reduce all those reduce operations?

hindol15:04:56

I was trying it out in a REPL and I reached here,

(let [n     2
      words ["roses" "are" "red" "sky" "is" "blue"]]
  (take (count words)
        (partition (inc n) 1
                   (concat (map-indexed vector words)
                           (repeat [nil nil])))))
;; => (([0 "roses"] [1 "are"] [2 "red"])
 ([1 "are"] [2 "red"] [3 "sky"])
 ([2 "red"] [3 "sky"] [4 "is"])
 ([3 "sky"] [4 "is"] [5 "blue"])
 ([4 "is"] [5 "blue"] [nil nil])
 ([5 "blue"] [nil nil] [nil nil]))

hindol15:04:59

Next task will be to assoc words in each group.

Aviv Kotek15:04:27

this becomes harder than the imperative solutions, i'll give it a try

hindol15:04:53

for is a Swiss army knife.

(let [n       2
      words   ["roses" "are" "red" "sky" "is" "blue"]
      groups (take (count words)
                    (partition (inc n) 1
                               (concat (map-indexed vector words)
                                       (repeat [nil nil]))))]
  (for [group groups
        [ix x] group
        [iy y] group
        :when (and ix iy (< ix iy))]
    [x y]))

;; =>
(["roses" "are"]
 ["roses" "red"]
 ["are" "red"]
 ["are" "red"]
 ["are" "sky"]
 ["red" "sky"]
 ["red" "sky"]
 ["red" "is"]
 ["sky" "is"]
 ["sky" "is"]
 ["sky" "blue"]
 ["is" "blue"]
 ["is" "blue"])

🚀 1
Aviv Kotek12:04:54

@UJRDALZA5, some further implementations have performed on Zulip (#data-science) if you are curious about final results, nicely done imo.

hindol12:04:11

Zulip UI is a mess. Can you share a direct link? Don't know if Zulip allows it.

Aviv Kotek12:04:49

joinr, jack rushner last comments will do.

hindol13:04:59

The last one is neat!

hindol13:04:15

And quite readable too.

Aviv Kotek13:04:36

yep, really nice

Aviv Kotek14:04:41

interested in better practices for vectorized /matrix programming with functional & clojure, :)

teodorlu13:04:44

If I were to use matrices in Clojure, I'd start with `http://tech.ml.dataset` tech.datatype: https://github.com/techascent/tech.datatype

Aviv Kotek09:04:59

why should use that and not core.matrix / nehandethral ?

teodorlu16:04:36

I don't think core.matrix has seen much love lately. About Neanderthal/tech.datatype, I think it comes to more preferece. I tried Neanderthal first, and wasn't 100 % happy with the docs. With tech.datatype, I've been thoroughly impressed with @UDRJMEFSN's work. The results he produces, how he documents his work, his design and how he engages with the community. He is super active over at https://clojurians.zulipchat.com/#narrow/stream/151924-data-science

Aviv Kotek18:04:45

i'll give it a look (already invested in that core.matrix :\\ )

teodorlu19:04:54

Hope it works out for you 🙂

Aron16:04:32

I have a set of keys and for each key another unique set of keys, for this second nested level, I would like to be able to encode information on the shape of the data it can take if the key appears in a map. So for example {:other {:fields #{:other_description}} and I need to say about :other_description that it is string and it needs to be minimum this length, maximum that length.

Aron16:04:45

What is the best way to do this?

didibus16:04:25

Probably using Spec, or some of the other similar libs like Schema or Mali

Aron16:04:02

well, spec of course is out of the question 🙂

noisesmith16:04:11

why is it out of the question? your "of course" is a non sequitor to me

Darin Douglass16:04:50

spec (or other lib) seems like the exact fit for this ¯\(ツ)

Aron17:04:22

because spec is not for this

Aron17:04:38

spec is for the developer to check if their code is ok, not for user data to be validated

Aron17:04:58

I am happy to be corrected on this!

noisesmith17:04:09

spec is for validation as well, last I checked - see for example s/conform

noisesmith17:04:45

https://clojure.org/guides/spec#_using_spec_for_validation - spec for validation is a heading in the official guide :D

Aron17:04:45

I know but I have several people telling me right now on other channels that it's not appropriate to run spec elsewhere than in a development environment. Confusing 😞

Darin Douglass17:04:09

what channels? i'd be interested to see their reasoning

Darin Douglass17:04:34

spec is just a way to define how data should be structured. you can use that in whatever ways you'd like: dev ergonomics, data validation, etc

noisesmith17:04:42

anyway, my understanding was this use case is why s/valid? and s/conform are provided

Aron17:04:57

thanks for the link, I am reading it now

Aron17:04:06

also, first time I heard this was in this video https://youtu.be/Xb0UhDeHzBM at 43:31

Aron17:04:23

the slide also says 'no runtime checking!'

didibus17:04:02

It does runtime validation

Aron17:04:18

ok, i am happy to be corrected. i will use it 🙂

didibus17:04:17

For runtime validation you want to use s/valid? . And for development you want to use instrument.

didibus17:04:46

It's quite a multi-purpose system, so you need to pay good attention.of what context people are talking of it

didibus17:04:06

Cause you can use it to spec your functions and instrument them so they are checked for mismatch at development time. But you can also spec your user input/output and validate them at run-time to protect yourself from bad input and report errors back to the user

didibus17:04:33

And then you can also use it for generative testing, for mocking, etc.

alexmiller17:04:59

it is perfectly valid to use spec at runtime to validate data. it is not recommended to instrument functions at runtime

alexmiller17:04:35

for runtime use, s/valid? or s/explain are probably the best api functions to use, and maybe secondarily s/conform

Aron17:04:58

thank you

seancorfield17:04:40

If it helps, this blog post talks about all the different ways we use Spec in dev, test, and production: https://corfield.org/blog/2019/09/13/using-spec/

❤️ 5
Aron16:04:09

will check the others

amarus20:04:02

Hello, guys! I'm new in clojure world and I'm creating simple API service with compojure-api lib. I'm writing schema for validation and don't understand how to validate and coerce date in my model. Can you help?

(ns hs-crud.patient
  (:require [schema.core :as s])

;; code for validators 

(s/defschema Patient
  {:fullname  (s/constrained s/Str valid-fullname?)
   :gender    (s/enum :male :female :other)
   :birthdate s/Str  ;; todo add date validator
   :address   (s/constrained s/Str valid-address?)
   :rpn       (s/constrained s/Str valid-rpn?)})
That is how my schema looks now

jtth21:04:56

is the recommended library for accessing firebase via clojure (on the jvm)? or should i just hit the REST API with clj-http or something?

kwrooijen22:04:37

Maybe something like this? https://github.com/alekcz/charmander I'm not really familiar with Firebase. I just happened to watch this talk of the author https://www.youtube.com/watch?v=j57UbYFbI-U He also made some more libraries to target Firebase

💜 2
Mario C.23:04:03

If you have a function that takes in a string and does an operation on it but given a nil value, throws an exception. Is the preferred way to wrap the operations in a try-catch or to just check if the argument is a string and return nil ?

Mario C.23:04:34

assuming that returning a nil value is a valid return type

Mario C.23:04:56

I think I am going to just check argument type since a nil parameter will be common

alidlorenzo23:04:36

nil is also returned by default so you can check that a value does exist, i.e. (when (some? x) ..) before running your operation

phronmophobic23:04:21

you can also check out fnil, although I prefer alidlo’s suggestion

Mario C.23:04:57

Thanks guys! Yes ive been trying to find uses for fnil since it looks handy but I always default to the when check

hindol12:04:49

Think you can drop the some? too. (when x ...)

Jorge Tovar23:04:31

Hi! Im currently working in a big project in a Colombian Bank to make a digital and cloud native application (=~Greenfield), currently our stack is Java . AWS DynamoDB , SQS, ECS Docker, HTTP integrations with retrofit and CircleCI ❤️. They gave me the opportunity to suggest another tech stack (We have some bugs and some performance problems, too verbose) . Im currently reading Clojure for the brave and `I want to know what are all the components like Datomic DB, Leiningen, ETC to make CRUD microservices , with tokens security that can make HTTP request? I have to make a POC` . [Two colleagues will make the same exercise with Scala and Elixit but I think with my little knowledge that clojure is the best choice]

noisesmith23:04:44

your fastest startup is probably lein with the luminus template, which makes a bunch of opinionated choices bout libraries for a reset server stack (with optional cljs frontend)

noisesmith23:04:26

many people disagree with choices provided by luminus but nothing else is as full featured in a new project template

Jorge Tovar00:04:50

The microservices are restful we have a mobile application. I woukd like to see some articles and examples to move to the cloud like nubank

introom07:04:13

@UGMEQUCTV why suggest datomic? the added benefits is often not a necessity. dynamodb itself is enough. adding a datomic layer hurts the performance of dynamo and adds point of failure.

hindol08:04:33

IMO learning Clojure and Datomic at the same time might be challenging. From your comment, it seems you are writing just the ReST API bits and not a frontend. I think https://github.com/ring-clojure/ring is the easiest. Ring is a spec and there are many libraries that conform to this spec. The https://github.com/sunng87/ring-jetty9-adapter can embed the Jetty webserver in the JAR itself. As for the API bits, there are many many options but you can check out http://clojure-liberator.github.io/liberator/.