Fork me on GitHub
#beginners
<
2020-11-19
>
Eric Ihli01:11:51

I'm looking for a resource that can help me learn how to parse arbitrary binary data. Given a description along the lines of "Offset 0000, 32 bit integer, magic number. Offset 0004, 32 bit integer, number of rows. etc...", It would be a huge help to see an example of a simple program that parses a custom binary data structure and... kind of like http://www.gigamonkeys.com/book/practical-parsing-binary-files.html but in Clojure. Actually, I think I might just go with that and try translating it to Clojure.

noisesmith18:11:17

you could also check out gloss, which is meant for this sort of thing https://github.com/ztellman/gloss

st3fan01:11:30

Is there a nice way to do the following? I feel there is too much repetition going on …

(defn should-add-label [settings issue]
  (and
   (check-issue-status settings issue)
   (check-issue-assignee settings issue)
   (check-issue-milestone settings issue)
   (check-issue-labels settings issue)))

st3fan01:11:39

(every? #(% settings issue) [#'check-status #'check-assignee #'check-milestone])

st3fan01:11:38

Sometimes it helps to think out loud

hiredman01:11:34

There may be an every-fn or every-pred already in core

st3fan02:11:07

ah yes but the predicates take only one argument

hiredman02:11:53

Oh, wait it takes multiple args, but as ands to the predicate weird

hiredman02:11:51

Nope, confusing myself with all the anonymous functions, the returned predicates takes multiple args as expected

st3fan02:11:15

Ok I need to re-read that

0xclj04:11:59

I have POST /parse-reply {body :body} (compojure-api) The content of (str(slurp body)) is along the lines of:

"--xYzZY
Content-Disposition: form-data; name=\"attachments\"


--xYzZY
Content-Disposition: form-data; name=\"dkim\"

{ : pass}
--xYzZY
Content-Disposition: form-data; name=\"subject\"

Re: TEST RESPONSE
--xYzZY
Content-Disposition: form-data; name=\"to\"


--xYzZY
Content-Disposition: form-data; name=\"spam_score\"

0.011
"
^ A multipart-form with boundary --xYzZY Is there a way I can convert it into JSON or if I have to parse the string to get the values by name like "spam_score" or "to" email address, how should I go about it? I’m thinking of splitting the string with --xYzZY and follow the parse-split cycle further until I get the value (yet to verify if the fields are always returned in the same position for the parse-split cycle to work).

Ben Sless08:11:08

An aside: I recommend against using slurp in production unless the endpoint sees very little traffic. Json libraries like Cheshire and jsonista can parse byte arrays directly

👍 3
0xclj11:11:25

Thanks!

👍 3
hiredman04:11:53

There is a ring middleware to decode form data

0xclj06:11:31

Yeah. Referring the last comment response of: https://stackoverflow.com/questions/33824124/multipart-params-are-not-extracted-by-ring-middleware#comment56033214_33829549 Went through: but not sure how to get it working. Especially the handler:

(require '[ring.middleware.defaults :refer :all])

(def site
  (wrap-defaults handler site-defaults))
The service I’m working on, has no handler defined.

theSherwood06:11:52

Hi. I’m trying to create a clojurescript library that makes use of some javascript files. I”m using leiningen and :foreign-libs under :cljsbuild in order to bundle the js into the library. However, that doesn’t seem to make the js code available to the application code consuming the library. Could anyone point out to me the proper way to incorporate javascript files into clojurescript libraries?

pinealan08:11:04

Is there someway in clojurescript to define constant values during compile-time, like the https://webpack.js.org/plugins/environment-plugin/ plugin?

didibus08:11:48

Euh... What does that plugin do?

didibus08:11:19

You can define constants with ^:const which will inline the value if its EDN

didibus08:11:06

Otherwise, you can use macros to perform whatever compile time things you want

pinealan08:11:00

I was trying to do it with macro just now but the compiler wasn’t happy about it

# .clj
(ns my.core)
(defmacro getenv [key]
  (System/getenv key))

# .cljs
(ns my.core 
  [:require-marcors my.core])

(def ENV (getenv))
(defn hi [] env)
This causes cljs compiler to warn me
WARNING: Can't take value of macro my.core/getenv at line 4
WARNING: Can't take value of macro my.core/getenv at line 5
I know it’s a very hacky way of using macro as a function, but I’m not sure how else to pass the JVM environment during compilation into runtime

didibus09:11:19

I wouldn't say the use of a macro for this is hacky. Like if you want this functionality, macros are a great way to get it. Much better than the JS pre-processing that webpack plugin must have to do. Now, that error normally means you are passing a macro to something that expects a function. I assume that's not the real code.you try, cause you have typos and all in it. Macros in ClojureScript can be a bit tricky to setup right, and then I'd just double check you don't have a dumb mistake, like missing a parens or forgetting to pass an arg, etc.

didibus10:11:35

So it could be a false positive warning. Apparently is fixed in a newer version of figwheel

pinealan12:11:48

thanks for the link, I wasn’t using figwheel, but I’ll double check the macro usage

dpsutton14:11:14

There’s a typo in your require macros expression. Is that present in the source or just in slack?

Noah Bogart14:11:15

is there a way to load into a default namespace with deps.edn?

Noah Bogart14:11:32

in lein's project.clj, I can say :repl-options {:init-ns default.core} and when the repl starts, it loads the file and enters the namespace so I don't have to do anything extra

Noah Bogart14:11:08

in my project's root directory, when i enter clj, it loads into the repl with my dependencies in the classpath, but it doesn't load the current project's files, and I have to call (load "default/core") to load them before I can call (in-ns 'default.core)

Alex Miller (Clojure team)14:11:09

you can use the -e features of clojure.main to evaluate expressions before you start the repl:

% clj -M -e "(require 'clojure.set)(in-ns 'clojure.set)" -r
#object[clojure.lang.Namespace 0x5bf22f18 "clojure.set"]
clojure.set=>

Alex Miller (Clojure team)14:11:43

that can be bundled into an alias in deps.edn as well (requires a , instead of spaces to avoid some word splitting through the shell):

{:aliases
 {:repl {:main-opts ["-e" "(require,'clojure.set)(in-ns,'clojure.set)" "-r"]}}}
and then clj -M:repl

Noah Bogart14:11:29

didn't look at that page

Alex Miller (Clojure team)14:11:59

yeah, -i can be useful to load a .clj file containing whatever in advance too

theSherwood16:11:55

Any feedback on using javascript files in cljs library code? https://clojurians.slack.com/archives/C053AK3F9/p1605769012476600

st3fan16:11:31

What is the idiomatic Clojure equivalent of Python's if "foo" in ("foo", "bar", "baz"): ?

dpsutton16:11:28

(or (contains? {"foo" "bar"} "foo") ({"foo" "bar"} "foo")) you can use the function contains? or use the fact that sets are callable themselves

dpsutton16:11:14

eval both of those expressions at the repl to see the different values they return

st3fan16:11:19

oh sets are callable 🙂

Chris Emerson19:11:20

Beware of contains? because it trips most new people up at some point. It's behaviour is sometimes different to what you expect!

Chris Emerson19:11:41

contains? checks for the presence of a key, for sets it does what you would expect, since keys=values, and for vectors it checks whether the index is present (contains? #{:a :b} :a)            ;;=> true (contains? #{:a :b} :c)            ;;=> false (contains? {:a :b} :a)             ;;=> true (contains? {:a :b} :b)             ;;=> false (contains? [1 2] 1)              ;;=> true (contains? [1 2] 2)              ;;=> false

st3fan19:11:45

Right so you need a # in your example @dpsutton 🙂

Chris Emerson19:11:45

yep! (contains? {"foo" "bar"} "foo") => true (contains? {"foo" "bar"} "bar") => false

Chris Emerson19:11:22

but if you used a set instead of a map:

(contains? #{"foo" "bar"} "bar")
=> true

dpsutton19:11:34

oh right. my bad 🙂

Daniel Stephens19:11:34

(some #{:some :things :to :check :are :contained} [:some :vec :of :things]) I quite like this pattern, uses the fact that sets can be called and lets you check for more than one thing (if needed) existing in the coll, no idea on the performance but it's never been my bottleneck

Chris Emerson19:11:15

that's a great tip

☺️ 3
st3fan19:11:21

Yeah that is nice - because right now I do (some #(contains? (set labels) (:name %)) (:labels issue))

st3fan19:11:08

Oh I can't directly translate that but i can shorten it with that trick

st3fan19:11:04

(some (set ignored-labels) (map :name (:labels issue)))

👍 3
noisesmith19:11:49

I'd usually just define ignored-labels as a set

noisesmith19:11:40

also consider: (some (comp (set ignored-labels) :name) (:labels issue)) - some already traverses, you don't need to map too

st3fan19:11:38

That is really cool

noisesmith19:11:01

you can even do something like (def ignored-issue (comp (set ignored-labels) :name)) then do (some ignored-issue (:labels issue))

noisesmith19:11:24

or maybe "ignored-label" is a better name there

noisesmith19:11:15

I find a tendency to reduce line / definition count as if that were meaningful simplicity, where giving the right things the right names is the clearest approach

st3fan19:11:56

"more smaller functions" ?

dpsutton19:11:18

Also there are caveats if the set can contain false or nil when invoking the set

noisesmith19:11:32

not just that - because the one liner has the same number of functions - the question is what is exposed as a name

mathpunk19:11:49

I have some data with timestamps like "2020-11-19T17:32:32.277Z" and I think it would be a little nicer to transform that to our time zone, and/or "time ago" format. Near as I can tell the move is to use java.time directly, instead of the various wrappers I can find (which seem to be for Java 8 ). There are a rather overwhelming number of classes and I cannot figure out which one, if any, has a constructor that will take a string like I've got. Any tips?

Daniel Stephens20:11:16

(defn format-when [zdt-str]
  (let [zdt (ZonedDateTime/parse zdt-str)
        dur (Duration/between zdt (ZonedDateTime/now))
        secs (.getSeconds dur)]
    (if (< secs (* 5 60 60))
      (let [[v a b] (->>
                      [[(quot secs (* 60 60 24)) "day" "days"]
                       [(quot (rem secs (* 60 60 24)) (* 60 60)) "hour" "hours"]
                       [(quot (rem secs (* 60 60)) 60) "minute" "minutes"]
                       [(rem secs 60) "second" "seconds"]]
                      (filter (comp (complement zero?) first))
                      first)]
        (if v (format "%s %s ago" v (if (= 1 v) a b))
              "now"))
      (str (.toLocalDateTime (.withZoneSameInstant zdt (ZoneId/systemDefault)))))))
=> #'user/format-when
(format-when "2020-11-18T17:32:32.277Z[Australia/Sydney]")
=> "2020-11-18T06:32:32.277"
(format-when "2020-11-19T17:32:32.277Z")
=> "2 hours ago"
(format-when (str (Instant/now)))
=> "now"
this is some very rough code I dug out, did something similar to this when I was starting out with clojure, probably helpful for the interop more than the clojure!

mathpunk22:11:44

ooh, i did work out how to use ZoneId/systemDefault but i didn't know about Duration/between, thank you

Daniel Stephens19:11:39

ZonedDateTime/parse

🎉 3
Daniel Stephens19:11:25

Instant/parse would also work, but if you are messing with timezones the first thing you'd probably do from there is convert to a ZonedDateTime

noisesmith19:11:18

there's multiple classes usable, but they each have parse methods

hiredman19:11:19

you may also be seeing the instant tagged literal in a print out

mathpunk19:11:11

I don't think so, hiredman, the data I've got is not coming from a Clojure program

mathpunk19:11:34

thanks folks! i thought I'd be using a constructor (Like. so) instead of a method

noisesmith20:11:35

the difference is that the parse methods make you specify the format you are parsing, where a constructor wouldn't (not that it couldn't, that would just be weird...)

seancorfield19:11:26

@mathpunk If you would prefer a wrapper, the clj-time README links to several (as part of trying to stop people using clj-time these days!).

sova-soars-the-sora20:11:44

Hi, html5 audio element needs my server to send "Accept-Ranges" "bytes" and while the default ring file-response sends "Content-Length" $len the audio element still says Live Broadcast as opposed to showing the seek bar as needed

noisesmith20:11:11

this might be relevant - each response still uses Content-Lenth, which just describes that payload not the resource https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/206

sova-soars-the-sora02:11:19

Right on. Thanks for snooping, man! Guess what I did? Used full paths instead of relative paths -- now html5 audio elements with .m4a works on mobile safari and desktop safari. Also... what the effnet?

sova-soars-the-sora20:11:18

I guess the header is supposed to read "Content-Range" ? hrm

sova-soars-the-sora20:11:47

Apparently it needs 3 headers set:

Accept-Ranges:bytes
Content-Length:7437847
Content-Range:bytes 0-7437846/7437847
but it doesn't look like Content-Range is by default in ring/file-response...

sova-soars-the-sora20:11:12

So basically it looks like I gotta add the "Content-Range" header... but i'm not sure how to get the byte length of a file

sova-soars-the-sora20:11:45

oh. looks like (.length ) does the trick with http://java.io

sova-soars-the-sora20:11:45

opened a new issue @ ring ^_^

st3fan20:11:54

Maybe I am overthinking this, but what is preferred in Clojure land? pr / pr-number or pull-request / pull-request-number ?

st3fan20:11:11

"pr" is a pretty common term in github speak of course

st3fan21:11:10

Oh hm their API calls it pull_number actually .. interesting

noisesmith21:11:26

as far as I know "pull request" is a github UI concept, it's been adopted elsewhere but isn't part of git itself

noisesmith21:11:38

I think this is off topic though

noisesmith21:11:38

@st3fan for example the clojure.core project uses github as a mirror, but it's not how contributions are accepted, you'd use JIRA instead

Alex Miller (Clojure team)21:11:50

the github repo is the repo

Alex Miller (Clojure team)21:11:30

this seems to be a pervasive thing people believe but I'm not really sure why as it has always been this way (since moving to git)

st3fan21:11:21

Sorry my question is about naming not about contributing to Clojure 🙂

noisesmith21:11:11

right, OK - regarding naming, pull-request is the name of a github UI feature

st3fan21:11:04

i'm working on a github client - so that is ok, i don't have to hide that detail

hiredman21:11:13

https://lore.kernel.org/git/[email protected]/ the first instance of "pull request" on the git mailing list predates github by 3 years

Joe21:11:56

I'm writing a small application which models populations changing over time - a sort of zero player game, similar to game of life. I want to draw the output of the steps on a canvas (a grid with squares on it that fill and empty) with the output being redrawn with each step. Is there a go-to Clojure library for this? I was thinking of quil, but only because it's the only one I've heard of. Are there alternatives I should be thinking about?

hiredman21:11:03

have you seen rich's ants simulation?

Joe21:11:29

I've heard about it in the context of core.async, i didn't realize it had a drawing component

hiredman21:11:49

it predates core.async by a lot

Joe21:11:18

Oh, not async, state management

hiredman21:11:36

I am not sure if the original is around any more https://gist.github.com/michiakig/1093917 is maybe a copy, it renders using java's built in 2d canvas stuff

Joe21:11:25

That looks good enough to do what I want, thanks!

sova-soars-the-sora21:11:23

There's an awesome story in Surely You're Joking Mr. Feynman about how Richard Feynman had a line of ants in his apartment and bathroom. Rather than try and get rid of them, he started experimenting. For example, he put some sugar on a table and then used a coaster to "elevate/teleport" ants to the table. Sure enough, ants were able to figure out there was an elevator and they would start waiting on the coaster to get a ride. But they couldn't really figure out that the elevator went 2 ways. Very interesting. Also he used chalk to draw a line for each ant's path to the sugar in his bathroom. It was like a newton's linear approximation. First it was curvy and clunky, and over time each ant successively did a straighter-and-straighter line towards the sugar.

Joe21:11:07

Love that book!

🐜 3
st3fan21:11:10

I'm a really happy camper right now. I rewrote the backend for http://devbots.xyz in Clojure (from Go) and it is really nice compact code and ... 10x less LoC 7000 vs 700

clojure-spin 15
👍 9
😲 6
aw_yeah 3
st3fan22:11:09

Is it possible to programatically "load" a namespace? for example my bots nicely organized in their own namespace and right now I have code like this:

(def apps [needs-triage/bot needs-review/bot lock-issue/bot lock-pull-request/bot])
(def apps-by-id (into {} (map #(vector (:id %) %) apps)))
But it would be nice if I can just discover all devbots.bots.* namespaces at runtime ...

st3fan22:11:12

They do contain multimethods, so I think I need to fully "load" them before the multimethod dispatch works.

st3fan22:11:20

(not sure what the right terminology is)

Alex Miller (Clojure team)22:11:36

you can load dynamically via require (or load, but that's considered lower level), but discovery is not generally a thing you can ask a jvm to do

Alex Miller (Clojure team)22:11:29

you can inspect parts of your classpath as files/jar paths to "discover" things like that (and there are java libs that have support for that kind of thing)

Alex Miller (Clojure team)22:11:36

probably best way to handle is with some kind of runtime config that lists the ns'es to load

st3fan22:11:20

these namespaces are embedded in my app, so i can also require them in core and then iterate over available namespaces and filter on devbots.bots. - maybe that is a simple start

sova-soars-the-sora02:11:19

Right on. Thanks for snooping, man! Guess what I did? Used full paths instead of relative paths -- now html5 audio elements with .m4a works on mobile safari and desktop safari. Also... what the effnet?