Fork me on GitHub
#clojure
<
2019-05-14
>
markx00:05:59

Is it possible to use items from an lazy seq one by one in order?

markx00:05:43

For example, repeatedly wait for 1 second and print a number from (range)

rutledgepaulv00:05:36

yes. loop/recur with first/rest should support that

noisesmith00:05:55

or (run! (fn [x] (Thread/sleep N) (println x)) (range))

noisesmith00:05:25

where N is the magic amount of time to sleep between items

noisesmith00:05:53

or

(doseq [item (range)]
  (Thread/sleep N)
  (println item))

markx00:05:24

My real use case is I want to print a lazy-seq of bytes read from socket. (repeatedly #(.read (.getInputStream (Socket. "localhost" 8888))))))

rutledgepaulv00:05:41

do you mean to open the socket repeatedly? or only read from it?

markx00:05:23

oh my bad. I meant only repeatedly reading, not opening.

markx00:05:37

Oh! thanks!

noisesmith00:05:43

so you aren't actually sleeping, just a blocking read?

markx00:05:54

Yeah, a blocking read.

noisesmith00:05:19

yeah, I'd use run! if it wraps up nicely in one function, or doseq if you have multiple imperative steps

noisesmith00:05:40

(and doesn't make sense to abstract the operation as a function for any other usage...)

noisesmith00:05:12

doseq also accepts all the syntactic tricks that for does (but imperative / eager)

markx00:05:15

OK it works! I was previously evaling from vim that’s why it didn’t print anything and hang. I tried it in the repl and it worked as expected. 😌

markx00:05:29

Thanks all

ahungry02:05:25

Anyone ever get cljfx working with lein? I'm asking around in #cljfx but it seems pretty empty. I installed the clojars package dependency in project.clj and ran lein deps, but when I try to include the ns, it errors out

ahungry02:05:28

not found in classpath etc.

seancorfield03:05:59

What error do you get @m131 and what version of Java are you running?

seancorfield03:05:38

(! 1107)-> java -version
openjdk version "11.0.1" 2018-10-16
OpenJDK Runtime Environment 18.9 (build 11.0.1+13)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.1+13, mixed mode)

Mon May 13 20:11:03
(sean)-(jobs:0)-(~/clojure)
(! 1108)-> clj -Sdeps '{:deps {cljfx {:mvn/version "1.2.9"}}}'
Clojure 1.10.0
user=> (require '[cljfx.api :as fx])
nil

seancorfield03:05:12

Seems to work with Leiningen too

(! 1109)-> lein new app jfx
Generating a project called jfx based on the 'app' template.

Mon May 13 20:12:02
(sean)-(jobs:0)-(~/clojure)
(! 1110)-> cd jfx

Mon May 13 20:12:05
(sean)-(jobs:0)-(~/clojure/jfx)
(! 1111)-> vi project.clj 

Mon May 13 20:12:46
(sean)-(jobs:0)-(~/clojure/jfx)
(! 1112)-> lein repl
nREPL server started on port 57958 on host 127.0.0.1 - 
...
jfx.core=> (require '[cljfx.api :as fx])
nil
jfx.core=> Bye for now!

Mon May 13 20:13:18
(sean)-(jobs:0)-(~/clojure/jfx)
(! 1113)-> cat project.clj 
(defproject jfx "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url ""
  :license {:name "EPL-2.0 OR GPL-2.0-or-later WITH Classpath-exception-2.0"
            :url ""}
  :dependencies [[org.clojure/clojure "1.10.0"]
                 [cljfx "1.2.9"]]
  :main ^:skip-aot jfx.core
  :target-path "target/%s"
  :profiles {:uberjar {:aot :all}})

Mon May 13 20:13:22
(sean)-(jobs:0)-(~/clojure/jfx)
(! 1114)-> 

ahungry03:05:57

Thanks - let me test - I have used openjfx with openjdk version "1.8.0_212" just fine in the past, but perhaps i'm hitting an issue now. let me give your samples a try

seancorfield03:05:31

I've never managed to get it to work with openjdk 8, only Oracle JDK 8 and OpenJDK 11.

ahungry03:05:19

https://github.com/ahungry/ahubu - That was my clojure browser project, it used openjfx, but I didn't use any of the clojurey abstraction libs like fn-fx or cljfx - it basically loads up the openjfx portion via .java files and hooks into it on the clojure side

ahungry03:05:31

openjfx with openjdk 1.8 that is

ahungry03:05:50

Maybe some class path is hardcoded in cljfx and can't be found under the openjdk hmm

ahungry03:05:12

When you got it to work with openjdk 11, was that by building openjfx from source? or does it shop as part of 11? (or as a separate package like with openjdk 1.8?)

seancorfield03:05:00

openjdk11 does not include javafx -- cljfx brings it in as dependencies I think.

ahungry03:05:00

Thanks - let me give 11 a whirl

seancorfield04:05:35

@m131 I figured I'd just try it on Windows PowerShell too

ahungry04:05:12

ah, what a frustration

ahungry04:05:00

seems like I can't get any luck with arch linux and this setup using lein - I can get the require to work without lein using deps, maybe lein isn't needed

ahungry04:05:23

is there a way to have lein leverage deps installed from the deps.edn?

seancorfield04:05:34

@m131 There's a plugin for that -- lein-tools-deps I think it's called.

seancorfield04:05:24

FWIW, we started with lein in 2011 at work, switched to boot in 2015, and switched completely to clj/`deps.edn` last year and we haven't looked back.

ahungry04:05:06

thanks, maybe that's the best way forward for me as well, I'll have to spend some more time on deps.edn

ahungry04:05:18

does anyone in your environment use Emacs/Cider with that setup?

seancorfield04:05:08

We use Socket REPLs and plain text REPLs. I used Emacs for years but switched to Atom (with ProtoREPL at first, but with Chlorine now). My colleague @hiredman uses Emacs but I'm pretty sure he's stopped using CIDER/nREPL these days.

ahungry04:05:23

Thanks again for the advice

seancorfield04:05:53

I'm a big fan of the simplest possible tools that work and support an effective REPL-driven workflow.

lilactown04:05:49

I use Emacs/CIDER and deps.edn every day. it’s good!

ahungry04:05:49

Yea, this does seem nice - with nothing more than a core.clj, deps.edn and cljfx as a dependency, I have the sample working now. I guess the other things I've used in lein, clj can replace as well? (generating an uberjar etc.?)

lilactown04:05:40

Ehhh not quite

lilactown04:05:13

There are some libs out there that try to fill those various holes to different degrees

lilactown04:05:55

In my libraries, I use leiningen to publish. In my apps, I install CLJ on the host I’m running on

seancorfield05:05:31

@m131 We use (my fork of) depstar to build uberjars for production deployment. Then we run them with java -cp path/to/the.jar clojure.main -m entry.point

seancorfield05:05:06

I publish libs to Clojars using clj -Spom, depstar and a mvn deploy task.

seancorfield05:05:33

(and there's a tools.deps/`clj` tool for publishing to Clojars if you prefer)

cavan07:05:11

Hi, I had a doubt about core.async, (async/alts!! [channel-im-expecting-data (async/timeout 30000)]) What happens to the timeout channel if i get the data from the other channel? Does my thread get occupied for the next 30 seconds till the timeoutchannel emits something or is it picked up by GC immediately after the alts!! gets the 'winner' channel

dominicm07:05:18

I'm trying to preserve the type of a collection after calling map over it. My attempt of (into (empty xs) (map f xs)) doesn't work for vectors because it reverses the arguments. Is there any other good tricks for this?

ivana07:05:31

((if (vector? xs) mapv map) f xs) ?

dominicm08:05:27

That's probably sensible tbh :) although strictly speaking, (not= (type (list 1 2 3)) (type (map inc (list 1 2 3))))

ivana08:05:04

of course, lazy raccoon is not obvious raccoon )

ivana08:05:04

but at most cases it dont need to make him strict before needed )

markx08:05:42

Hi, related to my previous question about lazy-seq of bytes from a socket:

(let [in (.getInputStream (Socket. "localhost" 8888))]
  (run! handle-byte (repeatedly #(.read in))))
What’s the advantage of doing seq operation here over recursively handling bytes? Like
(let [in (.getInputStream (Socket. "localhost" 8888))]
  (loop [b (.read in)]
    (handle-byte b)
    (recur (.read in))))

kszabo09:05:51

There’s an overhead to the seq abstraction, which at this level is quite significant. If you want to escape procedural code, I advise you turn to transducers

markx10:05:57

Do you mean the second style, which uses loop and recur, is procedural? And why should I escape it?

markx10:05:41

The way I see it is, even it’s procedural, it’s just the byte-reading part, which has side-effect anyway. Most of my business logic is inside handle-byte, which can still be functional.

kszabo10:05:21

Yes, it’s not a big deal. Still, what you are doing can be represented as mapping over a bytestream (which also correctly handles when it eventually runs out, unlike the versions you posted).

kszabo10:05:11

you can easily implement a bytes-in based on this

greywolve09:05:24

Why are business use cases, like timestamps etc a bad fit for Clojure metadata?

mloughlin11:05:41

sounds like business use case is the definition of data, not metadata 🤷

greywolve15:05:27

what about transient business data?

emccue20:05:56

if it is transient and doesn't matter, then it doesn't matter and you shouldnt bother keeping it

emccue20:05:13

if it matters, then it is part of your data and it shouldnt be metadata

👍 4
art10:05:21

Hi! I am not sure it’s a right channel, but is there anybody who ever worked with flatbuffers from Clojure? Its Java API seems reasonable, however at first look it does not seem it can be clojure idiomatic enough. I can miss something tho.

Jakub Holý (HolyJak)11:05:05

Is tap> suitable for data? I send a map to it but REBL shows it only as a string, and only first N characters (30??) so it is useless as most of it is namespaced keys 😞 @seancorfield, you use REBL and tap>, do you know? Thank you!!! Update: Mouse over shows the whole map for a few seconds, which solves the problem, though I would prefer to be also able to inspect it in REBL's viewer.

seancorfield16:05:36

@holyjak Currently, tap>'d values in REBL just display as strings -- you can't datafy/`nav` into them which is a bit of a shame. I use tap> for debugging and being able to see output alone is valuable but, yes, it would be a lot more valuable if you could drill down into tap>'d values in REBL like you can with regular values from the REPL. @U064X3EF3

seancorfield16:05:58

It's also kind of annoying that tap> returns true/`false` instead of the value you are "tapping". Makes it harder to use in a pipeline (you end up with (doto tap>) instead of just (tap>) and you can only use it with -> not ->>).

Alex Miller (Clojure team)16:05:29

I think we have a CLJ ticket for that

seancorfield17:05:03

I don't see one @U064X3EF3 -- I searched for both tap> and REBL.

Alex Miller (Clojure team)17:05:28

Maybe we’ve just talked about it then

seancorfield17:05:41

There's https://github.com/cognitect-labs/REBL-distro/issues/20 about tap> failing altogether in REBL on certain JDK versions (due to a reference to a class that has been removed).

Jakub Holý (HolyJak)10:05:20

thank you , Sean! Let's hope it will be getting. better...

seancorfield17:05:02

Oh, I'm sure Cognitect will continue to improve REBL...

Alex Miller (Clojure team)11:05:58

Should be data and I believe there was talk about ways to inspect and def as it, but not sure where that is

lilactown14:05:52

@holyjak are you using nrebl-middleware?

borkdude15:05:31

on Windows GraalVM native I get a nullpointer for io/resource because (.getContextClassLoader (Thread/currentThread)) returns nil. What should I use instead?

borkdude15:05:08

it’s probably a bug in GraalVM, but maybe there’s a workaround?

colinkahn16:05:57

I’m writing a macro that uses eval to do a series of requires and then uses them inside the body (like (mymacro (:require [my.ns :as m]) (m/some-fn))). It works fine except if I use a namespace aliased keyword (like (mymacro (:require [my.ns :as m]) (m/some-fn ::m/my-kw))). It throws an error saying it can’t resolve m. Curious if this is possible to get working and generally when are namespace aliased keywords resolved?

noisesmith16:05:27

namespace aliasing is done at read time

noisesmith16:05:56

which happens before your macro runs

borkdude16:05:06

I believe this is known as the Gilardi scenario

colinkahn16:05:05

Thanks @noisesmith, that makes sense. @borkdude reading…

colinkahn16:05:05

@borkdude interesting article, and i’m already using a wrapping do (my macro is based off of the one found here https://github.com/mfikes/chivorcam/blob/master/src/chivorcam/core.cljc). AFAICT though, as @noisesmith said, the namespace aliased keyword expansion is happening before anything gets eval’d.

mrchance16:05:25

Does anyone have experience with Jib for Clojure containers? What would be required to get that into boot or lein?

colinkahn16:05:39

Which I guess means that the compiler is analyzing the file using the requires in ns perhaps to do that namespace resolution?

noisesmith16:05:04

@colinkahn one work around would be something like the macro turning m my-kw (or anything else that isn't an aliased keyword) into ::m/my-kw on output

noisesmith16:05:38

it could be symbols, strings, whatever

noisesmith16:05:22

it could even convert :m/my-kw into ::m/my-kw - if that makes any sense for the macro internally

colinkahn16:05:38

Sort of implementing my own version of the expansion

noisesmith16:05:03

yeah - it really depends on what your macro is meant to do

colinkahn16:05:10

for this specific case I can work around it by just avoiding those namespace aliased keywords (it’ll be used in a fairly limited way, so that’ll be possible)

noisesmith16:05:37

also, there's nothing stopping you from just using :my.ns/some-key - is never needed, just convenient - what you said

colinkahn16:05:51

right, I did that initially too

acim117:05:47

Wondering if anyone can help me find a more elegant way to do this. So for this puzzle here https://adventofcode.com/2017/day/18 I've got a simple parser function to translate an instruction string into an instruction function. It's pretty boiler-platey though. In the end, I want to try multiple regexes to find what I need (with capture) and I want to short-circuit when I've got it (not try all the matches). I don't mind the repetition within regex strings, but the repeated when-let is pretty ugly:

noisesmith17:05:35

that could be cond

noisesmith17:05:59

oh, but the destructuring - you want a match type thing that has destructuring built into the clauses

noisesmith17:05:19

you could do the regex as #"([a-z]+) (\d+)" then you can use cond on the a-z

noisesmith17:05:26

or even case

acim117:05:44

Sure. I had considered some trick like that, but I guess I was hoping for something a bit more generic for this kind of thing. Not the end of the world, guess I could invest in some macro or something if I do this enough (suprisingly a lot in these AOC puzzles).

noisesmith17:05:35

I think one parsing rule then dispatching on data is better than N parsing rules

dpsutton17:05:08

doesn't match work with regex?

noisesmith17:05:24

match isn't in clojure.core

dpsutton17:05:05

ah. if that's part of the desire then i suppose its closed off

noisesmith17:05:30

usually with puzzles like these people aren't able to use deps? dunno

acim117:05:38

I can use whatever (no requirements, just for fun), but for sure I'll prefer things in core or more standard stuff when available.

dpsutton17:05:55

match is standard even though not core

acim117:05:48

Right. Can you see how it would fit in with my above example? If it's elegant and not too off the beaten path (not trying to refactor my whole solution into some prolog type thing) I'd for sure consider it

manutter5118:05:05

@mkeller02 what about something like this:

manutter5118:05:13

(defn parse-instruction [i]
    (let [[_ op x y] (some identity
                  [(re-matches #"(snd) (\d+)" i)
                   (re-matches #"(set) ([a-z]) (\d+)" i)
                   (re-matches #"(add) ([a-z]) (\d+)" i)
                   (re-matches #"(mul) ([a-z]) (\d+)" i)
                   (re-matches #"(mod) ([a-z]) (\d+)" i)
                   (re-matches #"(rcv) ([a-z])" i)])]
      (case op
        "snd" #(assoc % :snd (Long/parseLong x))
        "set" #(assoc % x (Long/parseLong y))
        "add" #(update % x (partial + (Long/parseLong y)))
        ,,,
        )))

manutter5118:05:25

Of course as soon as I write that my brain is screaming defmulti at me, but …

dpsutton18:05:27

i looked back and I just mapped read-string over the strings once they were split by space

acim118:05:21

^^^ Haha. That's another way.

acim118:05:19

@manutter51 I do like that better, works with the speculative destructuring

manutter5118:05:05

It kinda separates it into a lexing step and a parsing step, which I like

👍 4
noisesmith18:05:39

the code becomes quite a bit simpler if you do the regex once:

(defn try-number                                                                                                                                      
  [s]                                                                                                                                                 
  (try (Long/parseLong s)                                                                                                                             
       (catch Exception _ s)))

(def instruction-re #"(\w+) (\w+) ?(\w+)?")                                                                                                           

(defn parse-instruction                                                                                                                               
  [i]                                                                                                                                                 
  (let [[_ op a b] (re-matches instruction-re i)                                                                                                      
        x (try-number a)                                                                                                                              
        y (try-number b)]                                                                                                                             
    (case op                                                                                                                                          
      "snd" #(assoc % :snd x)                                                                                                                         
      "set" #(assoc % x y)))) 

noisesmith18:05:11

oh, I should have read the scrollback :D

noisesmith18:05:37

I guess speculatively trying to treat everything as a number in order to use only one regex is weird though :/

manutter5118:05:55

Looks like you could do it with 3 regexes though: op-digit, op-var-digit, and op-var.

noisesmith18:05:23

that's probably the most elegant version

markmarkmark18:05:00

you could split on spaces and have a defmulti that dispatches on first. then the defmethod can just take the whole space split line and do what it needs to do with it.

markmarkmark18:05:38

or even just a map of functions that are keyed by the instruction string

manutter5118:05:57

Yeah, that was what I was thinking as soon as I wrote out the some-based version

manutter5118:05:05

The nice thing about some + regexes is that you can specify that, for example, the add operator has to be followed by exactly a 1-character channel name and an integer value.

manutter5118:05:44

The multimethod would have to add extra logic to provide those constraints.

CyberSapiens9718:05:29

Hey guys, i'm creating a zip file for the client download with Ring. But i want after the user download's it, it automatically deletes the zip file, any ready solutions for that?

CyberSapiens9718:05:21

yeah i saw this but, deleting when the JVM exits is not a good solution

CyberSapiens9718:05:32

because it will be always online

CyberSapiens9718:05:48

it may be a solution by now, but not in the future

manutter5118:05:00

there’s a line at the bottom showing how to delete the file without waiting for the JVM to exit

CyberSapiens9718:05:42

yes, but i don't know how to trigger this delete

CyberSapiens9718:05:02

i don't know how to tell if a user have tried to access the file, and if he finished downloading or not

CyberSapiens9718:05:22

i'm trying to figure out something within Ring that does do this

lilactown18:05:02

you might consider deleting after a certain period of time. in case the user needs to re-try downloading it

CyberSapiens9718:05:21

yeah that's good too

lilactown18:05:38

something like 10 mins / 20 mins / 1 hr, whatever you feel is best

CyberSapiens9718:05:05

but in that case, there is any sort of lib that does this kind of job? runs a function every X minutes?

CyberSapiens9718:05:57

but the function would need to be specific to each created resource, because every resource has a different trigger time

CyberSapiens9718:05:18

wow, that seems to be hard to scale

lilactown18:05:53

there’s a few libs out there for running tasks on a timer. I’ve used https://github.com/juxt/tick

lilactown18:05:34

you could probably run it on a batch process: every 10 mins or so, run a function that checks if any files are over their expiration time and if so, delete them

manutter5118:05:41

I would probably do something like: have a task that checks the temps/ directory every few minutes, get the create-datetime for each file, and if the file is older than “X” minutes, delete it.

manutter5118:05:00

Lol, typing and not reading

manutter5118:05:24

The nice thing about periodically “sweeping” is that if your app crashes and restarts, it’ll automatically clean up after itself.

CyberSapiens9718:05:21

i'll check this ticklibrary and see if i can create the task, thanks guys

valtteri18:05:23

@cybersapiens97 if you need something solid and scalable I recommend pushing the zip-files to AWS S3 and programmatically generating download URL that’s valid for x seconds. It’s a pretty common use-case.

CyberSapiens9718:05:07

wow, that sounds nice, definitely i can possibly use this later on, after my beta phase is over...

CyberSapiens9718:05:17

but i'll have to study this later, because my zip files vary from 10mb to 200mb, and i don't want the user waiting to download for too long...

valtteri18:05:56

You don’t either want that your servers fill up from the zip-files and crash. 🙂

CyberSapiens9718:05:43

that's also true haha

kulminaator19:05:59

for the cases of "user has to download some generated content" - as my stuff is on aws i would definitely put the downloadable thing on s3

kulminaator19:05:10

and tell the user to get it from there, maybe directly, maybe via a redirect

kulminaator19:05:19

streaming big stuff out to possibly slow clients from your appserver is just looking for trouble 🙂

valtteri19:05:46

Yep, it’s a good idea to offload all the nasty bits to Amazon. 😉

noisesmith19:05:42

@cybersapiens97 what if the zip never had to be a file? you can provide a download that's simply bytes generated into your output

noisesmith19:05:14

this isn't something that ever needs to touch a file system

noisesmith19:05:52

(or worst case you can use a file system that's just in-process heap memory - which guarantees cleanup when the process exits anyway)

g22:05:56

hey everyone. i’m trying to work out a really simple example of parallelization to see if i can get some performance increase. is there any way i could potentially parallelize (apply min-key second {1 2 2 3 3 4 .. })?

g22:05:35

my first thought was something like splitting the list into n components and then running pmap across them, but i feel like the overhead would dominate

seancorfield22:05:11

@gtzogana How big is that map likely to be?

g22:05:35

let’s say on the order of 1000s

g22:05:02

this function would likely run super hot however, so any improvement would help

seancorfield22:05:57

clojure.core.reducers will probably help you here -- r/fold can do work in parallel.

g22:05:27

cool, thanks. i guess this is a good excuse to dive into that library

petterik22:05:39

@gtzogana using val instead of second is ~2x faster on my laptop:

(def m (zipmap (range 1000) (range 1000)))

(require '[criterium.core :refer [bench]])
(bench (apply min-key val m))
;;  Execution time mean : 41.214976 µs
(bench (apply min-key second m))
;; Execution time mean : 100.534521 µs

g22:05:19

wow. good find, thanks

Alex Miller (Clojure team)22:05:13

second walks it through the key, val is an indexed lookup

Alex Miller (Clojure team)22:05:31

and particularly special-cased for map entries

👍 4
souenzzo23:05:41

Why last is slower then val ?

souenzzo23:05:08

val performs like #(nth % 1)

seancorfield23:05:02

@souenzzo

user=> (source val)
(defn val
  "Returns the value in the map entry."
  {:added "1.0"
   :static true}
  [^java.util.Map$Entry e]
    (. e (getValue)))
nil

seancorfield23:05:24

val is a fast operation on a MapEntry

seancorfield23:05:15

You can't call val on an arbitrary two-element vector 🙂

souenzzo23:05:32

last is linear time 😮 I thought that last was constant time in counted structures