Fork me on GitHub
#clojure
<
2017-11-25
>
romain00:11:49

For those who use Docker, any reason to use java image instead of clojure image ?

noisesmith00:11:52

I've found making an uberjar (either on my machine or a build server) and then deploying as if it was java keeps things simple

romain00:11:38

I was thinking of a workflow including ansible & docker. Maybe it's better to just stick to ansible

Ryan Radomski00:11:11

Is docker more popular right now for production Clojure apps?

romain00:11:48

I saw some snippets to build & deploy with docker, but it seems a bit old and use java image instead of clojure one

Ryan Radomski00:11:46

I'm about to deploy for the first time, what is the "safe" bet?

noisesmith00:11:29

the simplest thing is to make an uberjar, make sure there's a jvm on your server, and run it

Ryan Radomski00:11:08

I was thinking that docker container networking might be a better fit for me because I want a local database connection. How do Clojarians typically handle that sort of situation?

noisesmith00:11:01

I've used dedicated database servers, so can't offer anything on that

noisesmith00:11:44

though with mongo, we do have a mongo server process for sharded dbs, that's set up by chef in our case

noisesmith00:11:15

(I think using ansible would be very similar)

romain01:11:34

🙂

Ryan Radomski01:11:00

This seems to confirm my suspicion that the benefits of solo UberJar is that simple deployment is easy, but doing anything non-trivial becomes a nightmare and the benefits of containers being that it can manage high amounts of deployment complexity, but is pretty heavy and frustrating to set up. Thanks for the resources, I'll have to ponder more what fits for my use case

noisesmith01:11:18

"nightmare" is not my experience

Ryan Radomski01:11:27

That's good to hear! Does the box running your UberJar have other dependencies that lein doesn't support?

Ryan Radomski01:11:57

I know you mentioned mongo, but I imagine you can keep the config inside of your .clj

noisesmith01:11:03

it has a few local deps that are set up by chef (I am not the one managing ops / chef stuff though)

noisesmith01:11:19

there's a mongo program that you need to run, it's not a client thing

noisesmith01:11:25

(for sharded dbs)

noisesmith01:11:12

basically it's a mongo server running on localhost that interacts with the sharded system, because the client can't use it directly

Ryan Radomski01:11:54

I'll have to consider the UberJar setup more seriously considering your positive experience. All of my professional dev is done on 'Nix boxes, and I'm a huge 'nix buff so I normally wouldn't shy away so much, but I have a few other portability considerations this time around.

noisesmith01:11:29

yeah, even just the differences between OSX and Linux are a pain, and I see how a container helps there

gonewest81801:11:45

Two startups in a row we’ve deployed services in docker and never regretted it. “Pretty heavy” and “frustrating” have not been our experience.

noisesmith01:11:48

but an uberjar is an uberjar, and it really does work

Ryan Radomski01:11:51

Thanks @gonewest818, I haven't heard of many people regretting dockerizing. I didn't mean to use such bold words! 🙂 I'm just trying to view my options for their upsides and their downsides.

Ryan Radomski01:11:46

I really don't think either option will cause too many problems. I'm thinking docker would give me a bit of the agility I'm after

Ryan Radomski01:11:24

Thanks much for the answers

romain01:11:26

@gonewest818 did you use java or clojure image ? just curious 🙂

gonewest81802:11:06

Custom images.

noisesmith02:11:39

regardless of the solution you pick, I would like to advocate for having a separate build server or process, that isn't something run on your production server (eg. let circleci create your uberjar and upload it, and download it from your server process)

gonewest81803:11:07

In our case we built the docker images too, and we tagged (w/ version numbers) and uploaded those to a local docker image repository.

cmal15:11:39

Hi, If I want to find the index of the biggest element that is <= 5 , and the index of the biggest element that is <=8, in a sorted array. Can I get the two indices in one search in Clojure?

cmal15:11:27

(defn tmp-search
  []
  (let [v [1 2 3 4 5 6 7 8 9 10]]
    (loop [index1 9
           index2 9]
      (let [condition1 (<= (get v index1) 5)
            condition2 (<= (get v index1) 8)]
        (if (and condition1 condition2)
          [index1 index2]
          (recur
           (if condition1 index1 (dec index1))
           (if condition2 index2 (dec index2))))))))

mfikes16:11:48

@cmal You could consider a one pass using reduce-kv

(reduce-kv (fn [[ndx1 ndx2] k v]
             (cond-> [ndx1 ndx2]
               (<= v 5) (assoc 0 k)
               (<= v 8) (assoc 1 k)))
  [nil nil]
  [1 2 3 4 5 6 7 8 9 10])

mfikes17:11:29

I suppose your approach is more efficient @cmal in that it traverses a part of the sequence from the end. You could force that into a single-pass reduce by using reduced to stop things. Thinking something like:

(take 2
  (reduce (fn [[ndx1 ndx2 ndx :as acc] v]
            (if (and ndx1 ndx2)
              (reduced acc)
              (cond-> [ndx1 ndx2 (dec ndx)]
                (and (nil? ndx1) (<= v 5)) (assoc 0 ndx)
                (and (nil? ndx2) (<= v 8)) (assoc 1 ndx))))
    [nil nil 9]
    (rseq [1 2 3 4 5 6 7 8 9 10])))

hmaurer18:11:05

Would you recommend using https://github.com/funcool/cats on a project, or are there better FP librairies out there? (for simple functor/monad stuff)

sundarj18:11:28

I've only heard good things about it. I know that @U0524B4UW is a fan

mccraigmccraig19:11:38

cats is ok @hmaurer - it was the best of the clojure fp libs i tried. the context stuff is a bit awkward, and i end up writing sugar macros for (with context ... (mlet [...

mccraigmccraig19:11:00

they took out the monad transformer stuff - i had to resurrect it for one of my libs (uses a promise-state transformer)

mccraigmccraig19:11:57

and there is no reader or continuation or comonad stuff

mccraigmccraig19:11:49

but what is there is solid and straightforward - i’ve come across only a couple of bugs in 2 years of heavy use in both clj and cljs

mccraigmccraig19:11:58

and if you use either to model failure you might also want to take a look at https://github.com/zalando-stups/cats.match

hmaurer18:11:04

Actually, my only real use-case right now is using Either to model failure instead of throwing exceptions

tbaldridge19:11:33

Since these sorts of solutions slow the code down, I’ve never used the either in production. But what’s the driver to avoid the exceptions?

hmaurer20:11:14

@U07TDTQNL I just thought it would be a neater way to differentiate between "expected" failure-like return values and actual exceptions

hmaurer20:11:45

for example, I am implementing a unification algorithm. When unification fails it's not really an error, just a failure scenario that the caller will handle

hmaurer20:11:52

a try/catch seemed a bit odd in that case

hmaurer20:11:12

I could return a custom map with a success/failure tag, but in that case why not use functors and get chaining etc for free...

tbaldridge20:11:01

Well one reason is that they are opaque. That’s my biggest criticism against using functions instead of data, there are few ways to introspect them.

tbaldridge20:11:47

But either way the chaining isn’t free, since either method requires using non standard Clojure forms instead of let, do, etc

tbaldridge20:11:07

But I agree exceptions are not the answer for this.

hmaurer21:11:18

@U07TDTQNL ah yes you are right, I was using the term "free" a bit too freely. I am open to better approaches!

dominicm14:11:53

This is something I've also been interested in, and has been one of the things I've felt myself wanting monads for.

hmaurer14:11:15

@U09LZR36F have you ended up using this approach on any project?

dominicm16:11:57

Not yet, no.

mccraigmccraig19:11:38

cats is ok @hmaurer - it was the best of the clojure fp libs i tried. the context stuff is a bit awkward, and i end up writing sugar macros for (with context ... (mlet [...

qqq21:11:24

is there a way to tell (for [[k v] someMap] ...) to process the elements in order sorted by 'k' ?

noisesmith21:11:15

@qqq (for [[k v] (sort-by key someMap)] ...)

noisesmith21:11:29

and, incidentally (sort someMap) will return key/value pairs in the same order as (sort-by key someMap) - when sorting collections it sorts by the size, then first item, then second item if first items sort equal

noisesmith21:11:13

I guess technically sort-by key might be different if sort thought two of your keys were equal even though hash does not

noisesmith21:11:00

fun

+user=> (assoc {} [:a] 0 '(:a) 1)
{[:a] 1}

tvalerio21:11:40

Is there a way to filter a collection with various rules? Fox example:

(def jobs [{:id "f26e890b-df8e-422e-a39c-7762aa0bac36",
            :type "type1",
            :urgent false}
           {:id "ed0e23ef-6c2b-430c-9b90-cd4f1ff74c88",
            :type "type2",
            :urgent true}
           {:id "690de6bc-163c-4345-bf6f-25dd0c58e864",
            :type "type3",
            :urgent false}
           {:id "c0033410-981c-428a-954a-35dec05ef1d2",
            :type "type2",
            :urgent false}])
I want to get the jobs with type 2, but considering this: return only urgent jobs if they exists. If not, return jobs with the same type that are not urgent. Just to avoid using lots of ifs… is there a way to do this with filter?

noisesmith21:11:37

@tvalerio (or (seq (filter :urgent jobs)) ...)

noisesmith21:11:58

oh, the type 2 part

noisesmith21:11:21

(let [type2 (filter (comp #{"type2"} :type) jobs)] (or (seq (filter :urgent type2)) type2))

tvalerio22:11:19

@noisesmith in this case type2 will have all the jobs right? Do I have to put the urgent part inside the let somehow?

noisesmith22:11:10

type2 will have all the type2 jobs, (or (seq (filter ....)) ...) ensures that you look at the regular jobs if none of them are urgent

noisesmith22:11:18

but otherwise you get the urgent ones only

tvalerio22:11:31

I tried in my application and it works perfectly! thanks

qqq22:11:51

What is wrong with this macro?

;; (comment java.lang.ClassCastException: clojure.lang.Symbol cannot be cast to clojure.lang.Keyword)

so the two 'small' examples work fine, but when I merge the maps, I get an error

qqq22:11:08

(defmacro def-fsi [args]
  `(do
     ~@(for [[v ks] (sort-by key args)
             k      ks]
         `(clojure.spec.alpha/def ~k ~v))))

(macroexpand-1
 '(def-fsi {number? [::a]}))
#_ (comment
     (do (clojure.spec.alpha/def :a.aa-util/a number?)))

(macroexpand-1
 `(def-fsi {::foo [::cat]}))
#_ (comment
     (do (clojure.spec.alpha/def :a.aa-util/cat :a.aa-util/foo)))

(macroexpand-1 
 '(def-fsi
    {number [::a]
     ::foo  [::cat]}))


Unandled java.lang.ClassCastException
   clojure.lang.Symbol cannot be cast to clojure.lang.Keyword

noisesmith22:11:28

@qqq is it number that it doesn't like?

qqq22:11:04

good cll; unfortunately

(macroexpand-1 
 '(def-fsi
    {number? [::a]
     ::foo  [::cat]}))
also give same error

noisesmith22:11:26

number? is also a symbol rather than a keyword

qqq22:11:31

what I don't get is why does

;; works
(macroexpand-1
 '(def-fsi {number? [::a]}))

qqq22:11:16

{number? [::a]} and {::foo [::cat]} works, but when I merge the two maps -- BAM, error -- can't even macro expand

qqq22:11:38

(sort-by key {'a 20 :fo 30}) -- totally your fault for suggesting (sort-by key ...) 🙂

qqq22:11:50

so clojure's sort can't compare keyword vs symbol

noisesmith22:11:12

you could provide a compare that knows how to

qqq22:11:38

I actualy doh't care about the order so long as it's deterministic

qqq22:11:43

maybe I'll just cast everything t strings

noisesmith22:11:40

@qqq (fn [coll] (into [(type (first coll))] coll)) - that ensures that every item in the collection has an item of type Class as its first element

noisesmith22:11:57

and the second elements would only be compared if the types of their first elements were the same, etc.

qqq22:11:27

does that get around this issue here: (sort [[:foo] ["hello"]]) ?

noisesmith22:11:14

hmm... it's not working as I expected...

noisesmith22:11:55

@qqq

+user=> (defn weird-compare [x y] (if (= (type x) (type y)) (compare x y) (compare (str (class x)) (str (class y)))))
#'user/weird-compare
+user=> (weird-compare :a :b)
-1
+user=> (weird-compare :a "b")
-7
+user=> (sort weird-compare [:a "b" 'c 3])
(:a c 3 "b")
+user=> (sort weird-compare [:a "b" 'c 3 66 2 50])
(:a c 2 3 50 66 "b")

noisesmith22:11:13

its order is stable, and you get arbitrary order if they are not equal

noisesmith22:11:26

it's not quite right yet for k/v or collections yet

andy.fingerhut23:11:28

If you search for 'cc-cmp' on this page, you can find a function I wrote a couple years back that tries to be a 'universal comparator' between objects of different classes, sorting first by class names alphabetically, then by class-specific means between objects of the same class: https://github.com/jafingerhut/thalia/blob/master/doc/other-topics/comparators.md

andy.fingerhut23:11:50

It doesn't succeed at being universal, quite, but it goes a lot farther than clojure.core/compare does.