Fork me on GitHub
#clojure
<
2018-02-14
>
qqq00:02:44

is there a clojure builtin for binary-slurp for dealing with binary files (and returhing a byte []) ?

noisesmith00:02:24

you can use Files.readAllBytes(Path)

noisesmith00:02:50

that requires some into-array and such in clojure, but it’s minimal, it works, and it uses stuff built into the vm

qqq00:02:37

@noisesmith: thanks; I've actually reailzed I should load it in chunks, as I'm sha-512-ing:

(defn sha-512 [fname]
      (let [md (MessageDigest/getInstance "SHA-512")
            ba (byte-array (* 1024 1024)) 
            fis (q/io-fis fname)]
        (loop []
          (let [len (.read fis ba)]
            (when (>= len 0)
              (.digest md ba 0 len)
              (recur))))
        (seq (.digest md))))

noisesmith00:02:39

user=> (java.nio.file.Files/readAllBytes (java.nio.file.Paths/get (java.net.URI. "file:///Users/justin/.profile")))
#object["[B" 0x531f4093 "[B@531f4093"]

qqq00:02:01

speaking of which, the loop/when/recur looks very ugly to me; suggestions to make taht more elegant welcome

noisesmith00:02:44

I’ve wanted a do/while version of loop (where the binding is created inside the “recur”, even on the first iteration)

noisesmith00:02:55

I guess that’s an easy macro to write

noisesmith00:02:40

@qqq am I reading that correctly, it only recurs if len is 0 or less? never mind - when not if

madstap00:02:46

@qqq Isn't that just a while loop? So something like:

(defn sha-512 [fname]
    (let [md (MessageDigest/getInstance "SHA-512")
          ba (byte-array (* 1024 1024))
          fis (q/io-fis fname)]
      (while (pos? (.read fis ba))
        (.digest md ba 0 len))
      (seq (.digest md))))

noisesmith00:02:04

well, len has to come from .read

noisesmith00:02:36

which is what made me think of my do/while idea

dpsutton00:02:04

someone code up a while-let macro

madstap00:02:35

(defmacro while-let [binding & body]
  `(loop []
     (when-let ~binding
       ~@body
       (recur))))

madstap01:02:16

I've never actually used while in code

noisesmith01:02:38

(ins)user=> (defmacro loop-while
               [bind condition & body]
               `(loop ~bind
                    (when ~condition ~@body
                        (recur ~(second bind)))))
#'user/loop-while
(ins)user=> (loop-while [t (java.util.Date.)] (> (rand) 0.1) (println t))
nil
(ins)user=> (loop-while [t (java.util.Date.)] (> (rand) 0.1) (println t))
#inst "2018-02-14T01:01:07.730-00:00"
#inst "2018-02-14T01:01:07.730-00:00"
#inst "2018-02-14T01:01:07.730-00:00"
#inst "2018-02-14T01:01:07.730-00:00"
#inst "2018-02-14T01:01:07.731-00:00"
#inst "2018-02-14T01:01:07.731-00:00"
#inst "2018-02-14T01:01:07.731-00:00"
#inst "2018-02-14T01:01:07.731-00:00"
nil

noisesmith01:02:52

of course that binding would usually reference t (and that actually works - just didn’t make sense in this silly example)

dpsutton01:02:08

i like the inclusion of loop in the macro name. you can't nest loops right? recur calls would be ambiguous?

qqq01:02:16

does while-let require read to return nil instead of -1 for exit?

noisesmith01:02:46

better example

user=> (loop-while [r (rand)] (> r 0.1) (println r))
0.7289088591417161
0.770206007468366
nil

noisesmith01:02:50

loop-while does! lets you use an arbitrary binding and condition

madstap01:02:30

nil or false, cause it just builds on when-let

qqq01:02:32

@noisesmith: are you releasing loop-while into the public domain? 🙂

noisesmith01:02:50

more general version

(ins)user=> (defmacro loop-while
               [bind condition & body]
               `(loop ~bind
                    (when ~condition
                       ~@body
                        (recur ~@(map second (partition 2 bind))))))
#'user/loop-while
(ins)user=> (loop-while [r (rand) s (rand)] (> r s) (println r s))
0.6569131819243965 0.2807053168342115
nil
(ins)user=> (loop-while [r (rand) s (rand)] (> r s) (println r s))
0.706471848833365 0.40927277078473123
0.23989683003342344 0.11729876666618255
0.9669516458014131 0.07665936063939205
nil

noisesmith01:02:40

the partition / map lets you put multiple bindings in the loop (as should be allowed)

noisesmith01:02:15

I’ve wanted this in a few places (usually where IO of some sort is involved) - glad to finally be motivated to write it

madstap01:02:19

Wouldn't it just use the innermost one?

noisesmith01:02:38

recur is always to the innermost loop

noisesmith01:02:49

(or fn, - recurrence target in general)

noisesmith01:02:43

perhaps it should be allowed to return a non-nil value, not sure where to put that in the syntax though

dpsutton01:02:02

oh ok. i wasn't sure if nested loops were possible. my bad

dpsutton01:02:23

i think i remember seeing a proposition for recur-to that looked interesting and went nowhere

noisesmith17:02:38

the issue here is that each function has its own invoke method, and you can’t do recur (a goto) across method boundaries

noisesmith17:02:57

perhaps there would be a way to do it with nested loops if they were all in the same method though

noisesmith17:02:03

yeah, notice there none of their potential solutions is tail call optimized

xiongtx02:02:17

Is there a 🐛 ticket to resolve this, or is this just how clj’s going to work?

(let [{:keys [foo/a foo/b]} {:foo/a 1 :foo/b 2}] [a b])    ;; => [1 2] 👌

(require '[clojure.string :as str])

(let [{:keys [::str/a ::str/b]} {::str/a 1 ::str/b 2}] [a b])    ;; => [1 2] 👍

(let [{:keys [clojure.string/a clojure.string/b]} {::str/a 1 ::str/b 2}] [a b]) ;; => [1 2] 👍

(let [{:keys [str/a str/b]} {::str/a 1 ::str/b 2}] [a b])    ;; => [nil nil] 😦

xiongtx02:02:58

B/c now there’s one scenario where you must destructure w/ keywords instead of symbols.

madstap02:02:09

Well, you can use the {::str/keys [a b]} syntax, so you don't need to use keywords

xiongtx02:02:18

But what if I have {::str/a 1 ::str/b 2 :foo/c 3}

madstap02:02:31

Then you could use {::str/keys [a b], :foo/keys [c]}. Kind of cumbersome though. I agree with you in that I'd expect all you examples to work...

xiongtx03:02:24

@U064X3EF3 Should the case I noted work? If so, I’ll create a JIRA ticket.

xiongtx03:02:11

Maybe not. I guess if str/a matched ::str/a, it’d be indistinguishable from :str/a, which it’d also match.

eriktjacobsen04:02:38

This doesn't look like a bug to me. It seems the double-colon is shorthand for namespace expansion, such that ::str is expanded to :clojure.string only in the context of there being an alias. Given that, ::str/a matches :clojure.string/a because it is expanded before checking the match, but str/a should not match :clojure.string/a.

Alex Miller (Clojure team)04:02:41

Right. This is correct behavior.

qqq03:02:44

which name is more clojuric ? sha512 or sha-512 ?

seancorfield03:02:27

@qqq Given that it's normally referred to as SHA-512 out there in the literature, I'd go with sha-512 in Clojure.

tomaas11:02:32

hi, this is monger docs :connections-per-host (default: 10) The maximum number of connections allowed per host for this Mongo instance. Those connections will be kept in a pool when idle. Once the pool is exhausted, any operation requiring a connection will block waiting for an available connection.

tomaas11:02:28

i'm confused of is it host in this context? Is this the server creating a connection?

tomaas13:02:32

hrrr, again asked before checking more that 2 min

tomaas13:02:38

there's ns.repl file

tomaas13:02:41

sorry 🙂

iGEL14:02:50

Hi! I'm confused by uberjar. When I run ring lein uberjar, I get a NullPointerException in some part of my code. I get why it's nil (because I read an envvar that's not set while I execute this command), but I don't get why my code is executed in the first place during compilation

joelsanchez14:02:48

are you using def for that?

joelsanchez14:02:07

wrap that in a function

iGEL14:02:46

def is executed during compilation? 😲

joelsanchez14:02:35

not an expert in compilation but, yes, the System/getenv call will be done at compile time

iGEL14:02:54

wow, that's unexpected

iGEL14:02:58

I'll try, thanks

joelsanchez14:02:24

all the code is run but with defn it's not a problem because the functions are not run, only the definition of them is run

joelsanchez14:02:33

that's why you should wrap it

joelsanchez14:02:13

same would happen if you did anything directly in the ns, try adding a println, you'll see

macrobartfast14:02:17

is there a cli tool for putting the latest version of a dependency into project.clj?

joelsanchez14:02:27

https://github.com/xsc/lein-ancient lein-ancient will happily check if your deps are out of date, and update them for you

macrobartfast14:02:01

that's actually pretty awesome.

macrobartfast14:02:16

getting them in there in the first place is my main concern atm...

macrobartfast14:02:52

tired of having to loop over to clojars each time... I'm thinking of something like npm install --save which I've gotten used to.

macrobartfast14:02:55

looping over to the browser and clojars each time is a subtle flow breaker.

macrobartfast14:02:14

the same for figuring out the exact string to require each library in a ns, which vary from the name used to pull it into the project in project.clj.

joelsanchez14:02:08

I'd suggest you to build your own lein plugin that uses the clojars api: curl -H "Accept: application/edn" get the latest version from there and add it to project.clj take a look at lein-ancient source code for inspiration

macrobartfast14:02:10

that would be a good project, actually.

macrobartfast14:02:28

I'd learn a lot as well.

joelsanchez14:02:57

lein plugins are easy to develop, and you already have one that does half of what you want 🙂

macrobartfast14:02:11

great suggestion... I'll take a look.

genec15:02:56

does anyone have any advice on using clojure for cloud / serverless (either aws or azure) ? a google search turns up links that are a year or two old. thanks!

valtteri19:02:07

There’s some discussion also in #aws-lambda

genec15:02:56

would be great if I could find examples of using clojure / azure functions as I could use it (sneak it in) at work

genec15:02:14

@lmergen I haven't seen that, thanks for the link!

genec15:02:47

but the worry is it hasn't been touched in 2 years

genec15:02:40

I did find this, which makes me think I can use clojure to write azure functions, https://azure.microsoft.com/en-us/blog/announcing-the-preview-of-java-support-for-azure-functions/

lmergen15:02:46

yeah it seems like there was a craze a few years ago where everyone was writing serverless apps and libraries, which appears to have cooled down a bit now

lmergen15:02:30

amazonica has support for lambda as well: https://github.com/mcohen01/amazonica#lambda

lmergen15:02:40

(which is a wrapper around the official AWS libs)

lmergen15:02:03

if i would want to start out with clojure + aws lambda, that would be my starting point

logistark15:02:29

Hi all, i am going to start a new project and I was going to pick pedestal, but seems that the last commits were made 4 months ago, is this project still active?

the2bears15:02:51

Projects in Clojure tend to be more stable, so don't read into (and I emphasize) only 4 months.

eigenhombre16:02:33

@genec If you do go serverless with Clojure (and it’s definitely possible) with AWS lambda, one important trick is to use a CloudWatch event to keep the lambda warm (e.g. invoke the lambda every five minutes to keep its response times < 1 second). I do this with Terraform (which I also recommend), working example here: https://github.com/eigenhombre/trav/blob/master/tf/terraform.tf#L152

genec16:02:57

@eigenhombre thanks for the advice

qqq20:02:38

@eigenhombre: how is this beneficial in practice, in my understanding (from talking to aws business support); this would only keep one lambda instance alive, and other lambda requests (when all isting capacity is used) w9ould still get delays, i.e. lambda doesn't 'prewarm' due to load

opsb20:02:51

I've heard the term "specificity" in a couple of talks and it seems to be a commonly used in the clojure community. I'm not sure what it's referring to though, what are they talking about? 🙂

tbaldridge20:02:36

@oliver089 depends on the topic, can you be a bit more specific 😄

opsb20:02:37

I've seen it discussed in the context of clojure-spec and also in random dicsussions like this one https://www.reddit.com/r/programming/comments/3bnt8m/rich_hickey_javaone_presentation_clojure_made/

tbaldridge21:02:44

In general it just means that you can (or have to) be very specific about what you want. In the case of this reddit article they're talking about OOP methods being specific to the object on which they are defined. In Clojure conj works on a lot of different types. In general .put in java is specific to the type its defined on.

opsb21:02:11

right ok

tbaldridge21:02:32

But it can be a good or bad thing, depending on the context or speaker.

tbaldridge21:02:53

Someone could say that spec is better than Java types because it gives more specificity via stuff like pos-int? vs Integer

opsb21:02:27

yes, I can see that it would provide far greater constraints than the java type system (not particularly hard 😉 )

opsb21:02:17

and I see your point about having more general functions to work on more widely used data structures

andy.fingerhut21:02:44

@oliver089 Did you watch the video recording starting at the point given in the link by the first commentor in that Reddit thread? The example of httpServletRequest's Java API vs. Ring that begins around time 47:50 with the slide titled "(Program) Size Matters" seemed fairly clear to me on how he meant specificity there. Transcript is available here if you prefer reading to listening: https://github.com/jafingerhut/talk-transcripts/blob/master/Hickey_Rich/ClojureMadeSimple.md

opsb22:02:32

@andy.fingerhut yeah I watched it after I posted the question, it got to the punchline very quickly and yes it makes perfect sense.

opsb22:02:04

(It's a long time since I've worked in Java so the HttpServletRequest example brought back memories 😉 )

markbastian22:02:35

Hi folks, is there a graphql/lacinia channel?

markbastian22:02:43

Or is this a good place to ask?

markbastian22:02:07

I'll just shoot.... I'm developing a schema.edn and my return type is a 2D array that can be any json type (e.g. String, Int, Double). Anyone know if this is possible to specify with lacinia/gql? Closest attempt:

{
:unions
;Note that this doesn't work. Unions must be objects.
 {:anything {:members [String Int]}}

  :table
  {:fields {:data {:type (list (list :anything))}}}
}

markbastian23:02:52

Ah, I think I got it. http://lacinia.readthedocs.io/en/latest/custom-scalars.html. I think I could create a custom scalar type for that.

markbastian23:02:48

Not sure if this is the best way to do it, but something like this works:

(def schema
  {:scalars
   {:Date
    {:parse     (schema/as-conformer #(.parse date-formatter %))
     :serialize (schema/as-conformer #(.format date-formatter %))}
    :anything
    {:parse     (schema/as-conformer edn/read-string)
     :serialize (schema/as-conformer identity)}}})

Busy escaping the comfort zone23:02:25

Hey moderators (and clojureians) is it ok to publish clojure projects on this channel?

bronsa23:02:59

there's #announcements which might be better fitted

andy.fingerhut23:02:05

I doubt anyone would mind, but there is #announcements channel specifically for that and other similar announcements

seancorfield23:02:52

@narkisr And when you post in #announcements it's a good idea to also tell folks which channel to use for any follow-up