Fork me on GitHub
#clojure
<
2019-04-26
>
yuhan03:04:38

what's the best data structure for expressing a bijective map? A <-> B where the elements within A and B are unique but could overlap

yuhan03:04:08

I could maintain two separate hashmaps A -> B and B ->A which are inverse of each other, but that seems expensive and error prone.

yuhan03:04:13

constantly calling map-invert on a large map doesn't sound good, I'm concerned about the performance because I'll be using both forward and backwards mappings in a tight inner loop

souenzzo03:04:17

you can try #(into % (map (juxt val key)) %) But if you need to access you data from many ways, may be a good idea use #datascript

yuhan03:04:53

Thanks! A full database is perhaps too heavy for my needs, I'm just trying to express a bijective mapping / isomorphism between two sets

yuhan03:04:05

It needs to be frequently updated in an inner loop, so I can't be constantly regenerating the inverse mapping, and some sort of persistent data sharing would be good

souenzzo03:04:42

"Full database is too heavy" datascript subtitle: What if creating a database would be as cheap as creating a Hashmap? 😝

😂 4
yuhan03:04:48

oops! Will have to look more into it then - I've hardly dealt with databases before

yuhan03:04:20

https://gist.github.com/semperos/3835417 Just found this implementation based on two synchronized maps 🙂

dadair03:04:40

I’m having some issue with the use of pprint/cl-format throwing an exception when run from an uberjar: ClassCastException clojure.pprint.pfroxy$java.io.Writer$IDeref$PrettyFlush$4923d848 cannot be cast to clojure.pprint.PrettyFlush; anyone run into this and know how to resolve? (pprint require’d via (:require [clojure.pprint :as pprint]))

dadair03:04:18

uberjar is built with juxt/pack.alpha via deps.edn

hiredman03:04:24

Looks vaguely familiar, likely something reloading causing Pretty Flush to be redefined, so it is a new different class that doesn't match the proxy class

dadair04:04:47

Seems it was related to some timestamp issue in uberjars; updating to a more recent juxt/pack.alpha seems to have fixed it

seancorfield04:04:54

I ran into a similar problem with depstar until I fixed in my fork. The obvious default way of building JAR files is wrong. Who knew? 😞

vinurs03:04:28

is there any library that when i add deps in lein project.clj and it auto reload it so that i needn’t restart the repl?

yuhan04:04:57

you can use Pomegranate to hotload dependencies at the REPL

yuhan04:04:45

not sure if there's any tool to pick them up automatically from the project.clj

seancorfield04:04:41

@haiyuan.vinurs Boot lets you add dependencies to the REPL while you're working (using Pomegranate under the hood I think). The Clojure CLI is built on tools.deps and there's a branch of that which supports an add-lib function to add new dependencies dynamically. I have an alias for that in my deps.edn file.

markx08:04:14

Is there a way to start a program, like using lein run, but also start nrepl in the background?

markx08:04:29

Something like https://github.com/jimbru/lein-repl-run does, but without a plugin.

Adrian Smith10:04:08

are there any cool clients built around datafy and nav yet? (besides rebl)

souenzzo11:04:19

@markx lein run myns.dev where src/myns/dev.clj contains something like

(ns myns.dev
  (:require [nrepl.server :as nrepl]
            [myns.core :as core]))

(defn -main
  [& args]
  (nrepl/start-server 3333)
  (apply core/-main args))

souenzzo11:04:55

"Functions and data compose better then plugins and middleware"

markx11:04:02

@souenzzo Thanks! This is what I’m currently doing. Just curious if there’s a better way.

markx11:04:40

I spent quite some time learning what lein run and lein repl do before getting anything done. 😅

souenzzo11:04:52

@markx my myns.dev from the current project do: - start a #pedestal server (main app) - start a #nrepl server - run "yarn install" - start a #shadow-cljs server - start ~3 builds in shadow-cljs server - run a "yarn watch" command "forever"

markx11:04:00

I don’t see your code, but that’s a lot lol

souenzzo11:04:06

that about: you will need to start/stop things, maybe in some order, and you dont need to develop a plugin to do it 🙂

markx11:04:13

like if you don’t have a plugin, you just manually do it, right?

markx11:04:54

Now I have a simple TUI app, and I just added a nrepl server to it. Maybe later I’ll have to add a repl too.

souenzzo11:04:29

I'm more about:If I can call (lib/start) I dont need to search a lein-lib plugin

souenzzo11:04:26

#tools-deps, the new cli to start clojure, it does not even have a plugin system

markx11:04:42

So you prefer clj over lein?

👍 4
souenzzo14:04:26

You don't need to prefer Both have one optimal use-case I deliver a product that my client require that java -jar app.jar will work, it's way easier to do with lein But what I'm saying is that you dont need plugins.

emilaasa11:04:28

I’m trying to use grpc and protobufs with lein-protoc and I do get the simplest hello world thing working. I don’t understand how to include dependencies tho - our company protobufs use com.google.api.grpc/google-apis-common-protos for example and I can’t seem to include them for lein-proto

arbscht11:04:46

looks like it tries to find the relevant jar among the classpath, for each of the given source-deps

arbscht11:04:15

so I'd try to make sure your jar is in the classpath first (e.g. as an ordinary dep), then name it in source-deps

emilaasa05:04:22

Thank you I will try that!

emilaasa04:05:43

It worked perfectly thanks again 😃

👍 4
emilaasa11:04:51

There’s a https://github.com/LiaisonTechnologies/lein-protoc#usage :proto-source-deps vector but I don’t get how I point it to a jar

Tangible Dream11:04:09

I’m going through some clojure homework that is proving beyond me, would anybody be able to assist?

Ivan Koz12:04:26

@darclan28 why don't you just ask an actual question about clojure so we can brainstorm it together?

Tangible Dream12:04:57

This is in a Koans form

Tangible Dream12:04:43

(defn empty-set
  "takes any input element and returns nil"
  [_] nil)

(defn singleton-set
  "takes an element and returns a fun-set containing
   just that element"
  [el1]
  (fn [el2]
    (when (= el2 el1) el2)))

Tangible Dream12:04:21

complete the following…

Tangible Dream12:04:28

(defn union
  "takes two fun-sets and returns a set that is the set
   of all elements in either set1 or set2."
  [f-set1 f-set2]
  ;;TODO
  )

Tangible Dream12:04:59

so first create two sets

Tangible Dream12:04:49

(def f-set1 (singleton-set :happy))

Tangible Dream12:04:13

(def f-set2 (singleton-set :sad))

Tangible Dream12:04:49

I tried

(def u-set (singleton-set f-set1 f-set2))
just to see what would happen

Tangible Dream12:04:08

got wrong number of arguments as expected

Ivan Koz12:04:13

you implemented singleton-set function?

Ivan Koz12:04:48

@darclan28 if i understand it correctly, singleton set can be described as a = singleton set of a

Ivan Koz12:04:11

am i right?

Tangible Dream12:04:16

That sounds correct

Ivan Koz12:04:28

then implementation should be

(defn singleton-set
  [x]
  (fn [] x))

Tangible Dream12:04:00

If I interpret the function in the sample correctly, it will re-assign a new value but keep the original value if identical, which sounds unnecessary

Ivan Koz12:04:50

@darclan28 well we describing singleton set, not a pair right?

Ivan Koz12:04:04

pair must be λabf.fab

manutter5112:04:08

No, the “fun set” idea is a function that returns a truthy value for elements that are members of the set.

manutter5112:04:02

so the union of two singleton sets should be a function that returns a truthy value if the given element is either a member of the first singleton set, or a member of the second.

manutter5112:04:06

@darclan28 You’re going to need to write a defn that returns an fn. The fn needs to take one element and return nil if the element is not a member of either singleton set, or the element itself as a “truthy” result, if it is in either set.

Tangible Dream12:04:55

That sounds like verification of contents in multiple sets, But not the combining of two sets into a single set

Tangible Dream12:04:00

"takes two fun-sets and returns a set that is the set
   of all elements in either set1 or set2."

Tangible Dream12:04:31

So here is a contains for a single fun set

Tangible Dream12:04:23

(defn f-contains?
  "takes an fun set and el and returns true only if that
   elements exists in the fun set else false"
  [f-set el]
    (= (nil? (f-set el)) false)
  )

Ivan Koz12:04:53

(defn empty-set
  "takes any input element and returns nil"
  [_] nil)

(defn singleton-set
  "takes an element and returns a fun-set containing
   just that element"
  [element]
  (fn [target]
    (when (= element target)
      target)))

(defn union
  "takes two fun-sets and returns a set that is the set
   of all elements in either set1 or set2."
  [x y]
  (fn [target]
    (or (x target)
        (y target))))

Ivan Koz12:04:57

like that i guess

dpsutton12:04:18

the definition of a singleton set in this exercise is a membership function. In this exercise a set is merely a function that recognizes membership

Ivan Koz12:04:09

what am i missing then

Tangible Dream12:04:34

need to pass false not false nil

Ivan Koz12:04:09

so separate nil and false

Tangible Dream12:04:09

FAIL in (union-test) (core_test.clj:44)
true false sets
expected: (= false (u-set1 false))
  actual: (not (= false nil))

Tangible Dream13:04:21

So that answer, using or, is there a book or site that goes over this kind of topic? It doesn’t feel intuitive to me.

Tangible Dream13:04:05

I’d be happy to go over a reading assignment before the quiz.

dpsutton13:04:32

Or can’t work here

dpsutton13:04:49

It prevents sets from having the value false in them

Ivan Koz13:04:11

(defn union
  [x y]
  (fn [target]
    (when (or (some? (x target))
              (some? (y target)))
      target)))

Ivan Koz13:04:29

how can we do it better?

manutter5113:04:14

@darclan28 Ironically, I became familiar with this style of “set” programming by taking the “Functional Programming in Scala” course at Coursera.

Ivan Koz13:04:26

(defn union
  [x y]
  (fn [target]
    (if (and (nil? (x target))
             (nil? (y target)))
      nil target)))

Ivan Koz13:04:50

i'm out of options

Ivan Koz13:04:58

when not and also

dpsutton13:04:02

(defn union
  [set1 set2]
  (fn [element]
    (if-some (set1 element)
      element
      (set2 element))))
is how i would do it

Ivan Koz13:04:26

wasn't aware of if-some great to know

dpsutton13:04:32

i used it incorrectly though

dpsutton13:04:51

its a binding one. so just use (if (some? ..))

dpsutton13:04:18

my only thinking is if nil is an explicit case make it very clear

manutter5113:04:47

If set1 is a function (as opposed to being a clojure set), you don’t need some?. Just call the function.

dpsutton13:04:00

need to watch for the case when the set contains false

dpsutton13:04:21

which was the original failure

manutter5113:04:55

Ah, I was just re-reading the test, that’s right

manutter5113:04:02

The test results, that is

Alex Miller (Clojure team)13:04:03

don't put logical false values in sets ;)

💯 4
manutter5113:04:57

(if-not (nil? (set1 value)) ...

manutter5113:04:05

unit test is tricksy, my precious…

manutter5113:04:22

Actually, some? is the better choice here. I keep thinking some? iterates over a collection, but it’s really just “not nil”.

Ivan Koz13:04:39

but i think if-not nil? is more describing, for a person who doesn't know some?

Ivan Koz13:04:01

(defn union
  [set1 set2]
  (fn [x]
    (if (some? (set1 x))
      x (set2 x))))

(defn union
  [set1 set2]
  (fn [x]
    (if-not (nil? (set1 x))
      x (set2 x))))

(defn union
  [set1 set2]
  (fn [x]
    (when (or (some? (set1 x))
              (some? (set2 x)))
      x)))

Ivan Koz13:04:04

all three for history

manutter5113:04:06

Check those, though, I think you just want (set2 x) where you have x (set2 x)

manutter5113:04:25

wait, no I’m reading that wrong

Ivan Koz13:04:30

yeah we have two sets

jthibaudeau13:04:32

(defn union [set1 set2] 
  (fn [v] 
    (if (nil? (set1 v))
      (set2 v) 
      v)))

jthibaudeau13:04:37

does that work?

manutter5113:04:43

ever since I quit drinking coffee, I swear… :face_with_rolling_eyes:

manutter5113:04:11

@jamesthibaudeau Yeah, that should work just fine

Ivan Koz13:04:02

now i'm confused about (set2 v) being first form in if

manutter5113:04:50

The trick is that each of your “fun-sets” should return nil if the target element is not in the set. So what happens if your set is a singleton set consisting of the value false?

manutter5113:04:30

In that case, it should return false if you pass it false, meaning “The element false IS a member of this set.”

manutter5113:04:24

which is confusing, because we naturally assume false means it’s not a member.

Ivan Koz13:04:01

yeah now i get it

Ivan Koz13:04:09

funny thing, like an optical illusion 😃

ivana15:04:47

I think all these things comes from wide repeated suggestion in internet not to use contains?, cause it exists espesially for such cases

djtango15:04:51

Is anyone able to explain this:

(defprotocol P
  (foo [this a]
       [this a b]))

(deftype T []
  P
  (foo [this a]
    (println "foo a"))
  P
  (foo [this a b]
    (println "foo a b")))

;; the-ns=> (foo (T.) 1 2)                                                     
;; foo a b
;; nil
;; the-ns=> (foo (T.) 1)                                                       
;; Execution error (AbstractMethodError) at kw-get/eval6029 (REPL:1).          
;; kw_get.T.foo(Ljava/lang/Object;)Ljava/lang/Object;

Alex Miller (Clojure team)15:04:18

why are are you extending P twice instead of doing it once?

Alex Miller (Clojure team)15:04:38

(deftype T []
  P
  (foo [this a]
    (println "foo a"))
  (foo [this a b]
    (println "foo a b")))

Alex Miller (Clojure team)15:04:25

I think only the second of those was taking effect the way you had it written

djtango15:04:16

Misunderstood the syntax

djtango15:04:49

so presumably deftype stays on a given protocol until a new one is supplied?

djtango15:04:17

I mistakenly thought it was 1 pair of Protocol to method

Alex Miller (Clojure team)15:04:46

you can specify any number of methods under the protocol

kurt-o-sys19:04:17

I'm developing an application in which I need the user to provide mathematical formulae... I need to parse these formulae so they can be evaluated. Are there any libs doing something like that? So, I'd like to read some edn-string like:

{:vars [x y]
 :result (* x 5 (Math/sin (/ y 7)))}
(it can be infix notation as well, but just being able to read this in clojure would certainly be fine as well. I could use something like eval, but that's not very save 😛. 'parsing' it, would result in something like (fn [{:keys [x y]}] (* x 5 (Math/sin (/ y 7))))

kurt-o-sys19:04:45

Are there any existing 'math parsing' libraries?

Alex Miller (Clojure team)19:04:55

seems like you have valid Clojure there - no need to "parse", just read

Alex Miller (Clojure team)19:04:34

you could then write an "evaluator" that took a list and recursively applied the first op to the latter ops, but of course that's all eval is doing

noisesmith19:04:11

if you want a whitelist, you could use a table

`{+ ~+ sin ~#(Math/sin %)...}
and only accept symbol inputs with substitutions in the table (edit - Math/sin isn't first class)

Alex Miller (Clojure team)19:04:14

another way to go would be to rely on eval, but put your time into whitelisting

kurt-o-sys19:04:55

ok, let me try and see if I get somewhere. (Couldn't make it work now, but I might have done something wrong/weird 🙂 )

noisesmith19:04:56

you could eval after the whitelisting, or even run the substitutions as an interpreter

kurt-o-sys19:04:57

something like this?

(defn calc
      [form vars]
      (let [ops-whitelist `{+ ~+ sin ~#(Math/sin %)}]
           (walk/postwalk
             (fn [calc]
                 (if (sequential? calc)
                   (if-let [f (ops-whitelist (first calc))]
                           (apply f (map #(get vars % %) (rest calc)))
                           (throw (new Exception (str  "Cannot perform operation: " (first calc)))))
                   calc))
             form)))
running
(calc `(+ "val1" 7 (sin 0.5)) {"val1" 5})
seems to work fine 😛

kurt-o-sys19:04:54

ouch, naming, not really pretty (twice calc)

noisesmith19:04:44

I think the eval version is fine - the postwalk ensures the safety I think(?)

kurt-o-sys19:04:10

yeah... not sure how I would've done the eval thingy.

noisesmith20:04:26

I posted a working example with eval / whitelisted symbols in the other thread (the one under you saying you wanted to avoid eval...)

kurt-o-sys20:04:47

oh, cool, thx, I'll have a look!

kurt-o-sys19:04:36

@noisesmith what d'you mean with run the substitutions as an interpreter

noisesmith19:04:59

walk the input, look up the thing in function position, call it with the args once all are simplified

noisesmith19:04:14

and bail if you hit anything that isn't a number literal and isn't whitelisted

yuhan19:04:57

https://github.com/rm-hull/infix/ check this out, if you're looking for external libs

kurt-o-sys19:04:29

yeah, saw that one, was about to start trying it, but having infix wasn't the main goal 😛

noisesmith19:04:35

perhaps the most elegant thing is read the whole thing into lists of symbols/numbers, then tree walk and replace symbols / validate, then eval

noisesmith19:04:08

the tree walk could just throw if it sees functions or vars that aren't allowed

noisesmith19:04:08

or if you are OK with users hosing / abusing the machine (eg. it is their own machine) just eval :D

kurt-o-sys19:04:21

lol, well, the whole point was to avoid eval 🙂

noisesmith19:04:24

the amusing thing, is if we map from symbol to safe math op, it looks like we have single quote strings

(def math-ops
  {'+ +'
   '- -'
   '* *'
   '/ /})

noisesmith19:04:08

(edit: /' doesn't exist)

Alex Miller (Clojure team)19:04:28

you are inherently writing an expression evaluator. it's ok to use eval for that, this is like what a lisp is good for. :)

noisesmith19:04:28

(cmd)user=> (load-file "/tmp/mather.clj")
#'user/run-math
(ins)user=> (run-math "(+ 1 1)")
2
(ins)user=> (run-math "(+ (sin 1) 1)")
1.8414709848078965
(ins)user=> (run-math "(+ (System/exit 1) 1)")
Execution error (AssertionError) at user/whitelist (mather.clj:15).
Assert failed: the System/exit operation is not allowed
op

noisesmith19:04:45

(require '[clojure.edn :as edn]
         '[clojure.walk :as walk])

(def math-ops
  {'+ +'
   '- -'
   '* *'
   '/ /
   'sin #(Math/sin %)})

(defn whitelist
  [tree]
  (if (symbol? tree)
    (let [op (find math-ops tree)]
      (assert op (format "the %s operation is not allowed"
                         tree))
      (val op))
    tree))

(defn run-math
  [s]
  (-> s
      (edn/read-string)
      (->> (walk/postwalk whitelist))
      (eval)))

yuhan19:04:25

here's one that doesn't rely on eval:

(defn eval-math [env expr]
  (let [oops (fn [msg x] (throw (IllegalArgumentException. (str msg x))))]
    (cond
      (list? expr)   (apply (or (math-ops (first expr))
                                (oops "Function not in whitelist: " (first expr)))
                       (map #(eval-math env %) (rest expr)))
      (symbol? expr) (or (env expr)
                         (oops "Undefined symbol: " expr))
      (number? expr) expr
      :else          (oops "Illegal value: " expr))))

(eval-math '{x 1 y 2}
  '(* x 5 (sin (/ y 7))))
;; => 1.4092142606110496

noisesmith20:04:49

one advantage(?) of the eval version is that you get hash-maps and vectors as functions, it would be good to get confirmation that whitelisting symbols is enough to make using eval safe here

kurt-o-sys21:04:17

thx all, I have a few versions now 😛

plins19:04:34

is there a way to run a fixture before ALL tests (not the one in current namespace) after ALL tests? I need to spin up a few docker containers before the tests and shut them down afterwards, maybe I should use lein for it?

noisesmith19:04:44

or write a clojure program that defines a middleware around clojure.test/run-tests

plins19:04:05

im trying to put the current timestamp in a uberjar name, assuming I can run arbitrary clojure code inside project.clj (not sure its possible)

{:uberjar       {:omit-source    true
                   :aot            :all
                   :uberjar-name   (str "cohmeia_api_local_"
                                     (.format (java.text.SimpleDateFormat. "yyyy_MM_dd'_'HH_mm_ss")
                                       (.getTime (java.util.Calendar/getInstance))) ".jar")
                   :source-paths   ["env/prod/clj"]
                   :resource-paths ["env/prod/resources"]}
fails with java.lang.ClassCastException: clojure.lang.PersistentList cannot be cast to java.lang.String Its possible to do this and am missing something or there is no way?

seancorfield19:04:17

In project.clj you can escape code with ~ to have it evaluated.

seancorfield19:04:55

so

:uberjar-name ~(str "cohmeia..." ...)

plins20:04:48

thanks 🙂

sotrhraven21:04:01

So is it to better to do :use or to :require? Is use akin to :refer all?

Joe Lane21:04:49

:require, don’t use :use

sotrhraven21:04:46

Ok, @vemv will bookmark that site.

sotrhraven22:04:10

That Stuart is a well of knowledge

✌️ 4
sogaiu23:04:54

@sotrhraven if you haven't done so already, perhaps you will find grepping the clojure source for sierra useful some time 🙂