Fork me on GitHub
#clojure
<
2018-12-14
>
macrobartfast01:12:58

I have a big non-specific question/topic for which I apologize in advance... I am trying to get my project off my laptop into the cloud, and I'm finding the whole thing daunting... there's kubernetes and all its providers, AWS EC2 and Elastic Beanstalk, just renting a VPS... so many possibilities. Does anyone have suggestions for a workflow for how to develop locally and then deploy into a persistence backed situation without scp'ing jars around (although that's on the table, absolutely, too)... thoughts, suggestions, counseling?

flefik13:12:13

a little late on the ball ... but I think one of the easiest ways to get something online (with postgres) is to use heroku + terraform. Heroku let's you deploy with just a git push, but it's also really easy to the integrate with your CI/CD. Terraform is nice because it's super easy for you to set up a different environment or change to a different provider later. Heroku only really makes sense for smallish projects. It's also completely free to get going (both vm and database)

flefik13:12:08

I store my terraform state on Amazon S3 (the backend bit), but you don't need that and can store it locally too. All you need to do is set the environment variables and then run terraform apply and you'll have an account created in heroku that you can push a rails + react app too. Just add the heroku git as a remote and off you go.

codeasone15:12:42

Deploying to ECS with https://github.com/jpignata/fargate was effortless for me so you could start there. Just add a Dockerfile containing your uberjar and deployment becomes a one-liner.

codeasone15:12:35

Depending on your persistence requirements, you can go a long way with localstack, plus perhaps postgres and redis containers in a docker composition to create some parity with your AWS deployment.

codeasone15:12:05

Problem not the best forum for this question, but I hope it helps 🙂

johnj01:12:36

no database?

macrobartfast02:12:44

yes, postgres currently.

macrobartfast02:12:12

that makes things a bit more complicated, definitely.

johnj02:12:36

yes, backup requirements come into play if any

johnj02:12:48

AWS RDS and EC2 might the simplest way to start (but not the cheapest)

johnj02:12:25

it really depends on so many things...

macrobartfast02:12:37

that's the direction I've been going in... I was a bit dismayed at how quickly the costs can add up for a hobby project... RDS can be stopped for 7 days and then it automatically wakes up and starts running up a bill.

johnj02:12:05

AWS kills you on traffic, a digitalocean VPS with backups enabled and an attached volume for the database (with some rsync cron jobs) may be good enough (and way cheaper) depending on your case, if you don't mind setting up postgresql yourself and doing some little ops

johnj02:12:13

also, look at heroku

johnj02:12:42

they have a free tier

johnj02:12:23

and managed postgres

macrobartfast02:12:05

oh wow, I was already having sticker shock and I hadn't even thought about the traffic.

macrobartfast02:12:33

I'll look at DigitalOcean... great idea.

macrobartfast02:12:06

and they have a new kubernetes thing, which means I could run postgres from a docker container, theoretically. but the whole K8 thing could be a rabbit hole.

johnj02:12:17

start with the basics 😉

macrobartfast02:12:01

sounds good... I'll look at DigitalOcean... the postgres setup will be a learning thing there, but sounds like a solid plan. 😊

eccentric J03:12:49

Is there a Clojure core function that splits a list into two lists: 1. items that pass a predicate 2. items that fail the predicate?

eccentric J05:12:30

I ended up using partition by

langmartin05:12:42

has anyone run into a lein repl error out with Exception in thread "main" java.lang.AssertionError: Assert failed: transport-fn? It seems to be a result of the latest macos mojave point update

langmartin05:12:58

happened to me right after I did the update to macos 10.14.2 today. I've tried java 8 and 11

eccentric J05:12:27

@langmartin https://github.com/technomancy/leiningen/issues/2497 if you’re in a project you can add the nrepl dep to your profiles.clj, downgrade to lein 2.8.1 via lein upgrade 2.8.1 or git clone master branch and run lein bootstrap

langmartin15:12:54

@jayzawrotny thank you! that's it

matthias05:12:32

I noticed that all of the channels here only go back about five days. Are older messages archived somewhere?

eccentric J05:12:38

Oooh perfect thanks!

eccentric J05:12:31

(defn separate-dirs-from-files
  [files-list]
  (reduce (fn [[dirs files] file]
            (let [name (.getName file)]
              (if (directory? file)
                [(conj dirs name) files]
                [dirs (conj files name)])))
          []
          files-list))
Is there a better way to write that?

jaihindhreddy-duplicate06:12:32

Haven't run it but...

(defn separate-dirs-from-files
  [nodes]
  (let [g (group-by directory? nodes)]
  	[(get g true []) (get g false [])]))

jaihindhreddy-duplicate06:12:55

Also, @jayzawrotny split-with gets you a vector of stuff that succeeds until first failure, and stuff that fails until first success. It is equivalent to #(vector (take-while % %2) (drop-while % %2)). Code was more concise and readable that my description of it 😅

eccentric J06:12:49

Thanks for the explanation. That makes sense but the tricky part is getting the name of each node, that’s why I went with reduce, otherwise I would have to map over them again.

👍 4
jaihindhreddy-duplicate08:12:23

Totally mis-read your code. My bad 😶

eccentric J15:12:46

@UBU6QCSJH all good 😄 that’s why I was hoping to find a better solution as reduce often means having to really study the body to understand it.

👍 4
Daouda07:12:22

hey guys, how to use env file in my clojure project whithout extern libs please?

Daouda07:12:07

or a better way to pass env to my clojure project

eccentric J07:12:01

Probably wouldn’t be too difficult to write a function that reads each line of an env file, splits it by = VAR_NAME=VALUE, then populates a map. Then you can design a general function for retrieving data from that map or System/getenv but at this point you probably just reinvented the wheel. This solution may suit your needs though https://stackoverflow.com/questions/32323572/is-there-a-way-to-set-system-properties-in-leinegen

jumar09:12:56

It can be better to use EDN syntax directly in your file and then just use clojure.edn/read-string

👍 4
Daouda08:12:23

can you guys tell me here difference between lein-2.7.1-alpine-onbuild and lein-2.7.1-alpine please?

martinklepsch11:12:04

(first
   (filter #(do (println %) (even? %))
           [1 2 3]))
Due to batching this prints all three entries of the vector — what's the best way to achieve the same outcome but only print 1 & 2?

mpenet11:12:50

you can use one of the many unchunk fns around

mpenet12:12:36

better would be not to use a lazy-seq I guess, but maybe you don't have a choice

martinklepsch12:12:22

(reduce #(when (even? %2) (reduced %2))
          []
          [1 2 3])
This works

martinklepsch12:12:35

Not sure I fully understand what you mean by not using a lazy-seq, the vector isn't lazy but filter makes it lazy, no?

mpenet12:12:02

filter returns a chunked lazy-seq

mpenet12:12:09

filterv would return a vector

mpenet12:12:44

if you have more complex stuff going on per iteration you can also think about using tranduce + halt-when

mpenet12:12:57

but in you example it seems not necessary

martinklepsch12:12:57

interesting, didn't know about halt-when

mpenet12:12:22

an eduction would work too

mpenet12:12:24

(eduction (comp (map #(doto % prn)) (halt-when even?)) [1 2 3])

martinklepsch12:12:27

I forgot that filterv exists in the stdlib 😄 Are you suggesting that this should work?

(first
   (filterv #(do (println %) (even? %))
            [1 2 3]))

mpenet12:12:52

no, it will print 3 times

martinklepsch12:12:59

yeah ok, that's what I'm seeing too

mpenet12:12:42

(eduction (halt-when even?) [1 2 3]) is the nicest so far imho

mpenet12:12:37

then you need to feed it to something if you use it more than once otherwise you'd get bad surprises

martinklepsch12:12:43

hm, I can't get the eduction to return the value for which the predicate fn returns true

martinklepsch12:12:21

But thanks, I'll take a closer look at that 🙂

mpenet12:12:36

right, you want the value, not a coll with it

Alex Miller (Clojure team)13:12:13

halt-when takes an optional second arg fn

Alex Miller (Clojure team)13:12:22

That retfn gets both the halt-on val and the pred result

mpenet13:12:55

doesn't seem to work with eduction tho, but it's fine with transduce of course

mpenet13:12:33

(transduce (halt-when even? (fn [_ x] x)) conj [] [1 2 3]) -> 2

mpenet13:12:56

which kind of make sense given the ret value of an eduction, it's not the same purpose

Noah Bogart13:12:58

Anyone have a “transducers for dumb people” or similar? I’ve tried a couple times to get it and it’s not stuck

jaydeesimon14:12:48

I know what you mean and here's some resources that I found really helpful...

jaydeesimon14:12:55

Arne Brasseur has some good content at https://lambdaisland.com. It's paid content but in my opinion, well-worth it. Specifically, episodes 38, 42, 43, and 44

jaydeesimon14:12:59

There's also some good content from Timothy Baldridge at https://tbaldridge.pivotshare.com/home on transducers; also paid but worth it

Noah Bogart14:12:33

Thanks! I’ll check those out!

jaydeesimon14:12:48

Also, this article is great for a practical real-world usage https://tech.grammarly.com/blog/building-etl-pipelines-with-clojure

jaydeesimon14:12:12

and also Ghadi Shayban's lightning talk at the conj two conjes ago also helped things click for me https://www.youtube.com/watch?v=FjKnlzQfAPc

jaihindhreddy-duplicate16:12:17

I watched Rich Hickey's talk at strange loop and got it. Had to then watch Inside Transducers a couple of times to get stateful transducers.

Audrius14:12:46

hi I am trying such interesting dynamic dispatch but I have got some interesting compile errors.

(def backend
  (case database-backend-keyword
    :mysql 'importer.persistence.mysql
    'importer.persistence.dummy))

(def insert! backend/insert!)

(def delete! backend/delete!)
'importer.persistence.mysql and 'importer.persistence.dummy are namespaces. Please advise on why does not this work.

benoit14:12:46

@masta Your backend var holds a symbol, not a namespace.

benoit14:12:17

Try ns-resolve

benoit14:12:00

But if I had to implement pluggable backends I would use a protocol.

Audrius14:12:17

protocol is not that elegant...

gganley14:12:13

Is there any way to update lein to use clojure 1.10?

enforser14:12:15

what do you mean? In a project you can change your Clojure dependency to be a 1.10 version

enforser14:12:22

and lein will pick it up

enforser14:12:04

https://clojure.org/community/downloads The latest dev release is "1.10.0-RC5"

gganley14:12:04

like if I run lein repl it runs 1.10

gganley14:12:37

Thank you for the in-project dependency though

gganley14:12:57

can I do a ~/.lein/profiles thing?

stathissideris14:12:06

@seancorfield just saw your article on datafy/nav, very helpful and thanks! one thing wasn’t very clear though: how can you tell whether an id in a row or a result set is a foreign key to another table?

stathissideris14:12:21

please ignore me, I’m reading the source of src/main/clojure/clojure/java/jdbc/datafy.clj and it’s obvious what your plan is 🙂

stathissideris14:12:21

follow up question: would you recommend this functionality for DB traversal in prod, or is this just a dev tool for now?

seancorfield16:12:40

REBL is a dev tool and that example of datafy and nav is intended for dev -- there's an overhead of datafication and the "dual navigation" of navigation that doesn't, to me, sit well with a production context.

seancorfield16:12:43

That said, I can sort of imagine datafy and nav being used at a different abstraction level to do data traversal. Imagine datafying a PreparedStatement, for example, and getting back a representation of the SQL data, and then nav on that could produce a PreparedStatement to query for related data... Not sure that I'd want to work with data that way but...

stathissideris11:12:20

thanks, I get your point about cost. I think there are some legitimate use cases that fit well with that kind of traversal, but some well-written sql join would most likely be preferable in most cases

semperos14:12:55

@gganley Leiningen consults a :repl profile if you define one, and yes you can do it in ~/.lein/profiles.clj (for reference: https://github.com/technomancy/leiningen/blob/3aad5ec7eb7eb91e9c5beb41aecf097c5d819472/src/leiningen/repl.clj#L363)

gganley14:12:39

Oh I forgot I was getting this error ->

gganley14:12:15

when I run lein repl this happens for some reason

gganley14:12:48

Is lein repl supposed to use nrepl? I thought that was only a CIDER thing

Alex Miller (Clojure team)14:12:02

this is a bug in lein 2.8.2 that was just released

Alex Miller (Clojure team)14:12:12

downgrade to 2.8.1 for the moment

gganley14:12:11

brew switch leiningen 2.8.1, Thanks!

stuarthalloway15:12:49

I made an example showing (well, hinting at) the power of bespoke datafy/nav https://github.com/stuarthalloway/reflector

seancorfield17:12:10

Very nice! A truly small amount of code for something so useful!

emccue16:12:23

@jayzawrotny I think you want group-by and not partition-by

worlds-endless16:12:58

Is it possible for lein's project.clj to look at a config file for certain values, like :version?

borkdude17:12:11

What’s wrong with this macro?

(defmacro uninterrupted [h & tail]
  (when h
    `(when-not (Thread/interrupted)
       ~h
       ~(uninterrupted tail))))

bronsa17:12:41

should be (uninterrupted ~tail)

borkdude17:12:58

oh wait, I should splice tail

borkdude17:12:12

that too yes 🙂

bronsa17:12:27

the main thing is that you just can't use a macro in the "runtime" part of the macro definition

borkdude17:12:29

this should do it:

(defmacro uninterrupted [h & tail]
  (when h
    `(when-not (Thread/interrupted)
       ~h
       ~(when tail `(uninterrupted [email protected])))))

bronsa17:12:09

don't think you need the innermost when

borkdude17:12:28

when I don’t do the when, the last uninterrupted gets called with 0 args 😕

borkdude17:12:01

but now the return value is always nil 😕

borkdude17:12:36

a bit hacky to fix the return value:

(defmacro uninterrupted [h & tail]
  (when h
    `(when-not (Thread/interrupted)
       ~h
       [email protected](when tail [`(uninterrupted [email protected])]))))

borkdude17:12:02

(uninterrupted 1 2 3) => 3

Alex Miller (Clojure team)17:12:34

you might want to consider how to handle InterruptedException too - if one of the expressions is blocked on something interruptible, it will throw InterruptedException as well

Alex Miller (Clojure team)17:12:21

maybe it’s fine to just let that throw, but you could also catch/ignore. generally, you might think about how/whether to give feedback for if interruption occurred

borkdude17:12:53

I’m currently only cancelling futures in this piece of code and handling java.util.concurrent.CancellationException when reading them

borkdude17:12:27

but yeah, good idea

daveliepmann17:12:52

Glad to see another Vulfpeck fan in Clojureland, Alex. :the_horns::skin-tone-3:

Alex Miller (Clojure team)18:12:24

I’m planning to see them at red rocks in May :)

upside_down_parrot 8
metal 4
Alex Miller (Clojure team)18:12:52

Should probably take this to #off-topic though :)

4
tomjack18:12:33

huh. I never noticed...

user> (:foo (sorted-map "foo" "bar"))
Execution error (ClassCastException) at user/eval36355 (REPL:83).
java.lang.String cannot be cast to clojure.lang.Keyword
user> (contains? (sorted-map "foo" "bar") :foo)
Execution error (ClassCastException) at user/eval36357 (REPL:86).
java.lang.String cannot be cast to clojure.lang.Keyword

noisesmith18:12:52

@tomjack "foo" and :foo are not the same

tomjack18:12:17

indeed. I would've guessed nil and false

noisesmith18:12:15

oh - it's because sorted-map needs to do sorting! you can't sort :foo into "foo" "bar"

noisesmith18:12:19

now I get it

seancorfield19:12:05

That is certainly a bit surprising on the face of it -- but kinda obvious when you think about sorting. I guess you can always use sorted-map-by with a custom comparator that lets you compare strings and keywords (however you want)...

Alex Miller (Clojure team)19:12:49

we’ve had a couple tickets on this and behavior has changed in some cases over the last couple releases

Alex Miller (Clojure team)19:12:27

the behavior in the example above I looked at in context of 1242 (was originally in there I think, but that ticket was narrowed) - I ultimately decided the get/contains behavior matched Java’s behavior on sorted sets (throws) and thus was correct as is, but I think it’s pretty gray

🤷 4
Jacob Haag19:12:04

So i've create a hello world project with lein new ___ and then ran lein jar. After unpacking the jar I see that the testing files are not included. Would anybody know why this is?

Alex Miller (Clojure team)19:12:44

test files are usually not included in artifacts

jjttjj20:12:23

Sorry in advance for the vague-ish question, but if a library has to manage an external connection, is it a general good rule that the library probably needs some sort of logging mechanism (tools.logging as a decent clojure default?) included to allow users of the library to log things relating to errors and such involving that connection? I'm about to release a very small library and not sure if I can get away with skipping a tool.logging dependency. The alternative is probably to add an error handling function to be passed as an argument in the library's connection api right?

noprompt22:12:25

Is ^:static still a thing? I can’t tell from the answers on the internet. 😕

hiredman22:12:09

it is not still a thing

✔️ 4
isak22:12:23

is there any sugar for making keywords namespaced in the current ns followed by a suffix, then the key name? E.g., sitting in the foo.accounting namespace, and want to make a foo.accounting.user/age keyword?

ag22:12:22

I need to encode a logic of trying number of times wrapped into catch block. i.e. sending a request that may cause java.net.SocketTimeoutException and trying for 5 times until there’s no exception. Possible without storing results of the request in a atom?

ag22:12:19

doing something like this, doesn’t look pretty:

(let [resp (atom nil)]
  (loop [tries 5]
    (try
      (reset! resp (http-get url options debug? description))
      (catch java.net.SocketTimeoutException ex
        (reset! resp nil)))
    (when (nil? @resp)
      (recur (dec tries)))))

hiredman22:12:06

generally I find using trampoline to be the cleanest way to loop around a try/catch

hiredman22:12:14

without pulling in a library you can do something like (trampoline (fn f [] (try whatever (catch Exception e f ))))

ag22:12:26

hmm…. nice

ag22:12:22

how do you make the counter (number of tries) thing with the trampoline approach?

ag22:12:44

oh wait… I think I got it

noprompt23:12:21

It seems like the macroexpansion of if-not is lacking parity with when-not.

noprompt23:12:31

(macroexpand
 '(if-not foo
    bar
    baz))
;; =>
(if (clojure.core/not foo)
  bar
  baz)

noprompt23:12:48

Should just be

(if foo
  baz
  bar)

noprompt23:12:56

Why bother with emiting the not?

noprompt23:12:37

when-not OTOH emits what I’d expect

(macroexpand
 '(when-not foo
    bar
    baz))
;; =>
(if foo nil (do bar baz))

noprompt23:12:49

I’m not really sure that’s ticket worthy or not.

dpsutton23:12:01

this has to do with where there are do forms i believe. your swapping of the if and else forms is not equivalent

noprompt23:12:09

Yeah it is.

dpsutton23:12:17

ah i was thinking clojure had the do block around the else branch. that's other lisps

noprompt23:12:31

(= (if-not false :a :b)
   (if true :a :b))
;; => true

dpsutton23:12:45

that test case is not sufficient to disprove what i was thinking but you are correct

noprompt23:12:51

Unless there’s some logical property I’m missing here I can’t see how the macroexpansion I’m proposing is incorrect.

noprompt23:12:15

It’s easily provable.

dpsutton23:12:33

i was thinking of in a lisp where there's a do block on the else branch they would not be the same, right?

(if nil
    (message "it was true")
  (message "it was false")
  (message "second block of it was false"))

noprompt23:12:57

That’s not a valid if.

dpsutton23:12:59

that's what i was confused about and what your test case there would not highlight i thought

noprompt23:12:48

I’m saying if-not does not need to expand literally into (if (not ,,,) ,,,) it can just expand into if with the then and else arms swapped.

noprompt23:12:54

(defmacro if-not*
  ([test then]
   `(if ~test
      nil
      ~then))
  ([test then else]
   `(if ~test
      ~else
      ~then)))

(require '[clojure.test.check.generators :as tc.gen])
(require '[clojure.test.check.properties :as tc.prop])

(every?
 :result
 (tc.gen/sample
  (tc.prop/for-all [falsey (tc.gen/elements [nil false true])]
    (= (if-not falsey :a :b)
       (if-not* falsey :a :b)))
  1000))
;; => true

4
dpsutton23:12:52

i thought more lisps had an implicit do body on the else branch. that was my confusion. it seems to just be emacs lisp

noprompt23:12:18

Ah, yeah. That’ll do it. 🙂

andy.fingerhut23:12:18

Your alternate macro-expansion of if-not looks correct to me, and would save a call to clojure.core/not.

andy.fingerhut23:12:57

Although code like that can I think be potentially in-lined if it gets hot in the JVM, depending on a bunch of JVM- and code-specific factors.

noprompt00:12:18

Anyway to test that? I’d like to submit a patch but only if it’s worthwhile to do so.

andy.fingerhut00:12:15

criterium is a good tool to force code to be executed enough times to cause it to be JITed, as long as you do not have JIT disabled in the JVM, and then only start measuring the run-time of provided Clojure expressions after that warmup time is complete.

andy.fingerhut00:12:20

I do not know which expressions would be convincing to do benchmarking on, but maybe you could find some used inside of Clojure's implementation itself, or some widely used Clojure library using https://crossclj.info

noprompt01:12:57

Thanks a ton @U0CMVHBL2! I’ll start my investigation. 👍

Alex Miller (Clojure team)03:12:55

I’m fairly certain this has been filed before and was declined, but I don’t recall the details

Alex Miller (Clojure team)03:12:22

I looked, and didn’t find anything. There was a ticket requesting unless that was basically same as when-not and perhaps that is what I was remembering, not sure

Alex Miller (Clojure team)03:12:05

I doubt there is a perf difference - the jvm is pretty smart at optimizing these kinds of things.

Alex Miller (Clojure team)03:12:46

I don’t really think this is worth doing tbh (not against it, just don’t think it’s important)

noprompt05:12:44

Here’s what I found: https://gist.github.com/noprompt/9f373fef5b070ee977f28b9daf2fc6ef Seems like it’s marginally faster for the nil and true cases. If the patch would be rejected all the same though I won’t waste anyone’s time submitting a patch. 🙂

noprompt06:12:23

The expansion to code that does work it doesn’t need to, in my opinion, is still a shame even in the face of “the JVM is smart”.

☝️ 4