Fork me on GitHub
#clojure
<
2019-06-28
>
diego.videco03:06:37

If I have a lazySeq and do something like (nth 4 my-seq), then the first 5 elements of my-seq would be realized right? But if afterwards I do (nth 5 my-seq), will clojure compute only the next element or 6 elements?

noisesmith17:06:38

lazy-seqs are cached, and aren't re-evaluated

andy.fingerhut03:06:33

I am trying to use a library I wrote to try to generate drawings of what happens to the in-memory data structures to demonstrate this, but I am pretty sure that it will only compute the ones that haven't already been computed before.

andy.fingerhut03:06:06

You can create a function that returns a lazy sequence, with an extra "println" call in the function that generates each element, if you want a fairly straightforward way to do some experiments there.

andy.fingerhut03:06:51

Realize that 'chunked' sequences are an optimization in Clojure that interacts with laziness, such that evaluation of some lazy sequences happens in entire "chunks" of consecutive elements all at once, rather than only one at a time.

andy.fingerhut03:06:11

The chunk size is 32 elements.

andy.fingerhut03:06:09

Some expressions you can try out in a REPL to see it in effect:

(def lazy-seq1 (map (fn [x] (println "generating elem for x=" x) (inc x))
                    (range 100)))
(nth lazy-seq1 0)
(nth lazy-seq1 5)
(nth lazy-seq1 30)

Drew Verlee03:06:16

apologies for returning to the same topic (in case your keeping track of my questions 🙂 ) is there a general solution for splitting off some work to be done in a separate thread, then stopping that work based on a timeout? my naive perspective is that we can just stop the process, then garbage collect any work it did and that would suffice.

andy.fingerhut03:06:29

A fully general clean solution to that is tough on the JVM, I believe, because a thread can in general be entangled in all kinds of ways with others, depending on the code it is running. e.g. it could lock 17 locks, be reading from 3 network sockets and writing to a file on a file system, etc. If you know that the thread won't be so entangled and difficult to stop, then a general solution for that restricted case is probably easier to find, and yet is still reliable.

seancorfield03:06:52

"In general" you can spawn and thread and then "interrupt" it, but the process in the thread has to be "interruptible". In other words, your processes have to be well-behaved.

seancorfield04:06:21

You can "stop" a thread (but it's deprecated but it's an inherently unsafe operation, because of locks etc etc).

seancorfield04:06:46

This is part of the "concurrency is hard" aspect 🙂

andy.fingerhut04:06:10

I haven't finished reading this article to see how in depth it goes, but it looks relevant (just found via a search): https://dzone.com/articles/understanding-thread-interruption-in-java

Drew Verlee04:06:09

So is it better to just build the stop condition into the functionality itself, even if that means injecting that logic in? using a while loop for instance that checks every time, to see if it timed out.

seancorfield04:06:33

@drewverlee Yeah, your process basically has to be "well-behaved" and check something periodically to determine whether it should stop or continue.

seancorfield04:06:59

FWIW, we tackled a similar problem at work some time ago. We built the process as consuming a lazy sequence, and the code that generated the lazy sequence remembered when it started and only returned new elements up until that "timeout".

andy.fingerhut04:06:24

I am guessing if that thread went into an infinite loop while processing one element of the lazy sequence, that is simply outside of well-behaved territory?

seancorfield04:06:45

Yeah, the black hole of processing blindness 🙂

Drew Verlee04:06:18

thanks for the feedback everyone.

Daniel Hines04:06:49

I've got a problem I think spec would be perfect for, but I'm a total spec noob and have no idea how to spec it. I'm using pseudo set notation to describe the problem (read "m E S" as "m is a member of set S"). --- Given: There's a set of types, T, There's a set of attributes, A, There's a set of specs S There's a map such that every attribute has a corresponding spec from S. What is the spec for 2-tuples where: - The first element R1 is a vector of where every element e E T. - The second element R2 is a set of tuples in the form [e a v], where e E {1..length of R1}, a E A, and the value of v conforms to the spec corresponding to a. Any help is appreciated!

andy.fingerhut08:06:03

Sorry, not very experienced with spec myself. The #clojure-spec channel may give you a more focused place to ask your question, although maybe most folks are away from keyboard at the moment.

👍 4
Daniel Hines04:06:41

Just to clarify, everything in the "Given" clause can be defined statically.

theeternalpulse05:06:52

I'm looking to create a macro that uses the map destructuring form of a vector, so

(defmacro test-m [& args]
  `(let [(hash-map ~@args)
         (vec ~args)]))

(macroexpand '(test-m a 0 b 1))

theeternalpulse05:06:16

But I'm getting an error for a simple symbol in the binding form

theeternalpulse05:06:07

This is a smaller scale version of a larger macro I want to use, but can't get past this part

andy.fingerhut05:06:17

What do you want the macroexpanded code to look like for (test-m a 0 b 1)?

theeternalpulse05:06:38

let me put the real example

(defmacro with-args
  {:style/indent 1}
  [argv arg-bindings & body]
  (let [defaults (apply hash-map arg-bindings)
        bind-list (->> (partition 2 arg-bindings)
                       (map-indexed (fn [i v]
                                      [(first v) i]))
                       flatten)]
    `(let [(hash-map ~@bind-list
                     :or ~defaults)
           (vec ~argv)]
       ~@body)))
I want the input to be
(with-args ["127.0.0.1 3030"]
                  [hostname "127.0.0.1"
                   port "6060"]
                  (println hostname)
                  (println port))
and the output to be
(let [{hostname 0
         port 1
         :or {hostname "127.0.0.1"
              port "3030"}} (vec args)]
    (println hostname)
    (println port))

theeternalpulse05:06:10

basically given a vector of args, bind using the key/index destructuring for associative lists

pinkfrog06:06:45

can i see a function is defined in which file

pinkfrog06:06:49

given the var to a function?

andy.fingerhut07:06:18

Try an expression like this at the REPL, replacing the namespace and function name with one of your own: (pprint (meta #'my-namespace/my-fn-name))

pinkfrog08:06:25

thanks. that really helps

andy.fingerhut07:06:36

It should have keys :file :line :column, which have useful values if the file was read view a require, and I'm not sure which other ways, too.

andy.fingerhut07:06:26

They do not contain very useful values if you defined the function in the REPL session itself.

danieroux08:06:53

How can I tell gen-class (or even the Clojure compiler) to generate a .class file that will run on Java8, when I’m compiling from Java11?

jumar09:06:28

Clojure should compile to the Java8 compatible bytecode. Does it not work for you?

danieroux09:06:59

I’ll have to get back to my desk and confirm. This is what I know: hf.depstar.uberjar with java -cp uber.jar clojure.main -m spike.core works. mach.pack.alpha.one-jar with java -jar spike-aot.jar -m spike.core is upset with the bytecode given to it. I don’t think it’s a dependency jar causing this, and thank you for the hint.

iGEL08:06:54

Hi. I would like to consume a API, and I do have an official swagger definition. Can I somehow generate a client for that API in clojure, for example similar to https://github.com/cognitect-labs/aws-api ?

jeroenvandijk09:06:58

is there a way to update the dispatch method of a multimethod in a repl session?

jumar10:06:04

I think you can also use (def my-multi nil) to "undefine" defmulti

jeroenvandijk09:06:31

(redefining the multimethod doesn't seem to work)

jeroenvandijk09:06:20

I was trying to remember why I define multimethods as (defmulti some-name #'dispatch-some-name)

jeroenvandijk09:06:21

ok found something: (alter-var-root #'some-name (constantly nil)) and then redefine multimethod

jeroenvandijk09:06:48

as alternative to var dereffing this works as well:

(defmulti some-name #(dispatch-some-name %))

jeroenvandijk09:06:13

(which might be more efficient in some cases?)

borkdude14:06:36

is the escaping that pr-str does for a string (e.g. prepending quotes with a backslash) different or the same than JSON?

Michael Griffiths17:06:03

Not quite the same – I believe all the escapes defined in clojure.core/char-escape-string (used by pr machinery) are the same as how you’d escape those chars in JSON, but the only chars that must be escaped according to the JSON spec are \, ", and control chars. The JSON spec also defines hex escapes like \u005C which are not part of the EDN spec (but most readers do recognise them)

csd16:06:32

What is the intended use-case for tap> and add-tap? Do you keep references to tap> in production code and only add-tap from the repl / test? If tap> isn't meant for production code, what is the use case where I'd want to use it instead of just printing?

Joe Lane16:06:59

When you want to inspect and manipulate the tap>’ed data after the fact and you want the instance itself, at a point in time.

csd16:06:52

Would you be able to share an example of what you mean by this?

csd16:06:10

n/m dpsutton's comment addresses this

dpsutton16:06:04

I've used it recently. I make a function that just keeps the last 10 tapped items in a list and can inspect them as data rather than just having printlns coming out. Its been quite helpful

dpsutton16:06:52

Also, since it gets multiplexed you can hook up a pretty printer and also have it stored somewhere so you can see it and also interact with it

misha16:06:05

I have a tree (irregular depth; more wide than deep; irregular branch factor, but <~100) and 2 paths to 2 leaves within it. what is the most efficient way to get leaves (or their paths) between those 2 paths? clojure and clojurescript plz :)

andy.fingerhut16:06:47

So every node in the tree has an ordered list of child nodes and/or leaves?

misha16:06:39

yes, but path is not a part of the node. so when you are in the node, you don't know where it is at in a tree, unless you track paths while you traverse

andy.fingerhut16:06:57

So if you had a function to do this, its inputs would be (a) the root of the tree, from which everything is reachable, (b) leaf node 1, with no information about how to reach it from the root, and (c) leaf node 2, with no information about how to reach it from the root ?

misha16:06:02

think nested edn, rendered by react with (map-indexed (fn [idx child] ...) (:children node))

misha16:06:29

tree, path-from, path-to

misha16:06:59

the actual task, is to get set of nodes between start and end of browser DOM selection, hence you have path-from and path-to at that point.

misha16:06:21

the most straightforward solution I have, is to: 1) get-in tree common-ancestor 2) cut-out pre and post children from ancestor's :children vec 3) walk that subtree, doing things to leaves

misha16:06:44

yeah, going through zippers right now

theeternalpulse16:06:11

That's actually not the link I found useful, there was a video on it by a developer here on a video series

misha16:06:25

that's the video playing on background now, so it's nice to have some confirmation it is useful )

theeternalpulse16:06:21

haha, yeah, it is very cool since lips AST's are such easy trees to parse, and having that is very useful, especially if you've dealt with AST trees in most languages.

misha16:06:27

looks like prewalk of sliced tree is less code and more straightforward to read

misha16:06:01

inb4 Nathan's specter :)

Alan Thompson17:06:14

For general tree-like data processing, you may find the Tupelo Forest lib useful. There is also a video from the 2017 Clojure Conj: https://github.com/cloojure/tupelo/blob/master/docs/forest.adoc

theeternalpulse17:06:23

I guess Specter is a use case as well, it has quite a few selectors for changing in place. I guess zippers has a good use case for manipulating lisps itself, as I believe in the video by Arne he states it's used by some tools that format clojure in the repl.

theeternalpulse17:06:04

grocking specter can take some time, and upkeep as I always forget when I come back to it

Alan Thompson17:06:15

And be sure to see the API docs

theeternalpulse17:06:32

neat, thanks @cloojure, "better than zippers" is a bold statement

theeternalpulse17:06:39

I like bold statements lol

Alan Thompson17:06:42

Clojure zippers are broken in several edge cases involving the first & last child nodes. See the excellent blog posts from http://josf.info/blog/

borkdude17:06:28

@alexmiller why does your account come up twice when you type @al on JIRA? it's bit confusing, because I don't know if the top or bottom one is better 😉

Alex Miller (Clojure team)18:06:19

well, I actually have 3 accts because reasons. fixing this is a gigantic pain in the ass. fortunately, it doesn't really matter which you pick. ;)

borkdude18:06:25

is it reasonable to force clojure 1.10+ on user of a new library? I'm using (symbol :foo) which is only available in 1.10+

Alan Thompson18:06:39

Depends. I have a few functions that are just throw if the clj version is too old. If the user never calls that fn, they can use an older clj version.

andy.fingerhut18:06:05

If your library is open source released under same license as Clojure, you have the option of copying the definition of a function from Clojure into your lib. Depending upon what else you use, that might enable going back a Clojure version or two.

borkdude18:06:50

I can also optimize the code and just call .sym on the keyword

Alan Thompson18:06:52

Actually, I mostly use it to guard against older Java versions. The idea is similar to what core.async does if you try to use a feature outside the lexical scope of the go macro.

borkdude18:06:06

maybe there are also backport projects for clojure 1.8-1.9 like what clojure-future-spec was for clojure 1.8

andy.fingerhut18:06:38

A backport project, meaning something that adds to 1.9 everything that is in 1.10? I haven't heard of such a project.

borkdude18:06:58

yes, one that patches symbol for example and adds some predicates that might be new

borkdude18:06:20

I'd say just upgrade clojure, what's the big deal, but other people might think otherwise

Alan Thompson18:06:55

"...but other people might think otherwise".... In 2008, I was working on a project that still used Fortran-77. Not Fortran-95, or Fortran-2003, or Fortran-2008. Not even C or C++ (yes, even C++ would have been a huge advance compared to Fortran-77).

andy.fingerhut18:06:11

Q15 on 2019 state of Clojure survey shows 97% on Clojure 1.8 and later. See the chart if you want more detail: https://www.surveymonkey.com/results/SM-S9JVNXNQV/

borkdude18:06:20

Clojure is more easily upgradable than other languages

borkdude18:06:14

with a little bit of effort I could support clojure 1.9 I guess

Alex Miller (Clojure team)18:06:31

I would try pretty hard to avoid making backport projects. that is imo more painful than the pain you're trying to avoid.

borkdude18:06:46

yeah, also, it prevents people from upgrading 🙂

Alex Miller (Clojure team)18:06:42

choosing when to bump minimum deps does not have a hard rule and depends a lot on the lib.

borkdude18:06:11

I guess (symbol :foo) is the only change I'd have to support, I can't see any other things in the changelogs that should give problems on 1.9

Alex Miller (Clojure team)18:06:16

right now, I think requiring at least Clojure 1.8 (or even 1.9) should be uncontroversial. 1.10 is a half year old, so you are probably cutting off some user base by requiring it. whether that's ok is up to you

borkdude18:06:13

it is better to specify a lower clojure version in project.clj, so people don't get warnings for overriding the clojure version?

Alex Miller (Clojure team)18:06:40

I think it's best to specify the lowest version you depend on, ie the truth

borkdude18:06:30

for the next clojure survey it would maybe interesting to ask which OS people are using

borkdude18:06:41

as Windows support is a topic that comes up now and then

andy.fingerhut18:06:02

I'd guess the number of Windows users in that survey is a fairly small percentage, but note that friction between some existing Clojure tooling and Windows could be an influence there.

borkdude19:06:05

so a chicken and egg problem, or a cultural problem?

andy.fingerhut19:06:05

Maybe this is putting it too bluntly, but probably because the number of people in the intersection of the sets "cares about Clojure", "wants to use Windows as primary development platform", and "wants to spend time debugging and improving the tooling infrastructure" is very very small.

borkdude19:06:32

and the intersection of people not using Windows, but would like their tooling to be able to used on Windows, but also want to spend a lot of time and energy on that is even smaller

theeternalpulse19:06:48

can tools-deps fetch deps from a gist?

andy.fingerhut19:06:15

I am personally way outside the set "likes to spend time debugging and fixing Windows-specific IDE problems"

theeternalpulse19:06:02

this one didn't work well

with-args {:git/url ""
                   :sha "e21b6f7fed74dca079edf2806b71791a"}

theeternalpulse19:06:22

i tried with the whole url and it told me to use the sha

Alex Miller (Clojure team)19:06:32

well I've seen people doing it, so I know it's possible

noisesmith19:06:32

I'm pretty sure there's no gist support, unless there's a way to access a gist as a weirdly named git repo

theeternalpulse19:06:02

haha, I thought I saw it, but in tools like shadowclj or boot, but not deps

borkdude19:06:06

@theeternalpulse that gist should have a deps.edn

Alex Miller (Clojure team)19:06:50

it doesn't have to if you force it to be a deps project

Alex Miller (Clojure team)19:06:01

by adding :deps/manifest :deps

Alex Miller (Clojure team)19:06:07

it's tolerant of missing deps.edn

borkdude19:06:15

ah, is that new?

borkdude19:06:22

for me it is 😉

Alex Miller (Clojure team)19:06:23

has been like that for long time

theeternalpulse19:06:25

oooh, love it, trying

theeternalpulse19:06:48

for some reason I've grokked tools.deps more than any of the other tooling so far

theeternalpulse19:06:13

not sure where that gis tis getting the sha from

noisesmith19:06:32

isn't the sha part of the gist url?

theeternalpulse19:06:49

in the example, the sha and gisturl sha are different

theeternalpulse19:06:24

{hello-clojure {:git/url "" :sha "1c9a05106171f97f9b8e8ac8f58c7bd8dbe021f9"}}

borkdude19:06:15

I have a user of clj-kondo who uses keywords like :&::before to generate CSS. This style of keyword is allowed in CLJS, but tools.reader barfs on it:

(clojure.tools.reader.edn/read-string ":&::before") ;;=> 
Execution error (ExceptionInfo) at clojure.tools.reader.impl.errors/throw-ex (errors.clj:34).
Invalid keyword: &::before.
What to make of this?

noisesmith19:06:14

tools.reader has always been stricter about keywords than the clojure reader is

borkdude19:06:19

btw this is allowed works:

(clojure.tools.reader.edn/read-string ":&:before") ;;=> :&:before
so it seems to be about the double colon

Alan Thompson19:06:40

@misha What is the question?

misha19:06:01

that's what I end up writing for: https://clojurians.slack.com/archives/C03S1KBA2/p1561738205383400 where you responded with forest url

Alan Thompson20:06:37

1. I'm not sure what children you are trying to find....? 2. Always starting at the root?

misha20:06:41

need to update leaves between 2 paths (including leaves at those 2 paths)

misha20:06:09

well, starting at the root, instead of common ancestor, because tree is really shallow, and it does not matter much, but good point (if that's what you ment)

Alan Thompson20:06:24

So you want to find the node at path [1 4 1], then apply f to that and every following node in DFS order, right?

Alan Thompson21:06:26

Here is how to do it in plain clojure.

misha21:06:43

1) not just "after", but "between" p1 and p2 2) your approach walks all nodes, and if tree is huge, but p1 p2 path-range is narrow – you walk entire tree anyway. 3) atom is kinda smell here 4) what is tupelo? tupelo and append is not "plain clojure" I'd think. 5) use in ns is very bad 6) you consume stack with this kind of recursion

misha22:06:03

anyway, thanks for participating :)

andy.fingerhut19:06:14

Is that perhaps a difference between tools.reader.edn and tools.reader?

andy.fingerhut19:06:13

edn is more restrictive by design, but good for reading data from untrusted sources. full tools.reader should be limited to data from trusted sources (which linting your own code and libraries you use, should be).

andy.fingerhut19:06:06

@borkdude At least with tools.reader 1.3.2:

user=> (clojure.tools.reader/read-string ":&::before")
:&::before

andy.fingerhut19:06:55

Does clj-kondo use the edn reader to read code? I would guess that would hit edn limitations for lots of code bases.

borkdude19:06:27

@andy.fingerhut I'm using rewrite-clj which uses tools.reader

borkdude19:06:35

for parsing some things like keywords

andy.fingerhut19:06:28

At least the first paragraph of rewrite-clj's README says it is restricted to reading edn subset.

andy.fingerhut19:06:05

but I am not familiar with its internals

borkdude19:06:33

so far I've hadn't had many issues except with nest namespaced keywords, which was easy to fix

borkdude19:06:15

hmm, using normal read-string works, I'll see how that behaves then

theeternalpulse19:06:16

I found doing git ls-remote returns the commit shas

borkdude20:06:35

btw, the keyword isn't accepted by clojure itself:

user=> :&::before
Syntax error reading source at (REPL:5:0).
Invalid token: :&::before
so it might not be unreasonable to say: use (keyword ...) if you want to fabricate weird keywords, but don't use keywords that aren't accepted by Clojure, not even in CLJS, because that will make your code more future proof (e.g. when converting to .cljc)

wilkerlucio20:06:30

hello, I'm working on some code to wrap the Jena RDF API with Clojure, I was trying to use the clojure protocols to make a Jena Resource to act like a map, is this possible? I'm trying following this tutorial: https://gist.github.com/blacktaxi/1687594, then I have something like this in my code:

(extend-type Resource
  clojure.lang.IPersistentMap
  (assoc [this k v]
    this)
  (assocEx [this k v]
    this)
  (without [this k]
    this)
  ...)
But when I try to load it complains that clojure.lang.IPersistentMap is not a protocol (it is an interface). Given this, there are other protocols in Clojure I can use to make some third part defined class to act like a Clojure map?

noisesmith20:06:04

@wilkerlucio if you use composition instead of inheritance, you can define a deftype where the instance has a reference to a Resource and uses it for the apropriate interface methods

wilkerlucio20:06:18

@noisesmith yeah, that's the direction I'm thinking now, makes sense, then I can have the rest of the functions to expect this instead of the direct class reference

noisesmith20:06:25

@ztellman’s potemkin library https://github.com/ztellman/potemkin has a utility for defining new "map like" types

csd21:06:11

If more is implemented in terms of next, why is it fair to say that "`rest` is lazier than next" (e.g. https://stackoverflow.com/questions/4288476/clojure-rest-vs-next)?

andy.fingerhut21:06:40

What is more in your question?

csd21:06:18

e.g. in ASeq

andy.fingerhut21:06:41

I could be missing something, but perhaps rest can be said to be "apparently lazier" as observed only by its return value, but they are equally strict/lazy in their under-the-cover implementations ?

hiredman21:06:28

ASeq is one possible implementation

hiredman21:06:40

checkout LazySeq

csd21:06:53

LazySeq's implementations of next and more are nearly identical except next can return nil. Not sure how that is lazier. https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/LazySeq.java#L79

hiredman21:06:56

that is exactly how it is lazier

hiredman21:06:42

the difference between returning nil if there is no more stuff vs. returning a another lazy-seq that may or may not be empty

hiredman21:06:28

Cons is maybe a better thing to look at

csd21:06:45

maybe this was more pertinent to LazyCons

hiredman21:06:59

lazycons doesn't exist any more

csd21:06:10

i mean the guidance that i'm seeing on the net

hiredman21:06:31

when lazy-cons did exist there was only rest, and rest would never return an empty seq, if there was nothing more you would get nil

hiredman21:06:04

The implementation in Cons(which is typically what lazy seqs are built on top of) makes it clearer

hiredman21:06:27

next returns more().seq(), and more just returns _more

csd21:06:58

ah that makes sense

csd21:06:00

can you clarify the relationship between LazySeq and Cons, though? i see there is a cons method on LazySeq, but internally nothing uses it or otherwise references Cons

csd21:06:30

I guess the core functions like map etc are passing a cons into lazy-seq when they return

hiredman21:06:01

lazy-seq is macro that wraps its body in a thunk and passes that thunk to the LazySeq constructor

hiredman21:06:31

that thunk when invoked returns a seq

hiredman21:06:51

typically it calls clojure.core/cons to construct that seq

hiredman21:06:16

clojure.core/cons takes something and a seq and constructs a Cons

csd21:06:12

Maybe this is outdated guidance on clojuredocs https://clojuredocs.org/clojure.core/next. https://clojure.org/reference/lazy has similar guidance

hiredman21:06:13

lazy has a note at the top that says "Treat this page as a useful historical record of these design decisions, rather than as reference documentation."

csd21:06:53

noted. i'm wondering whether either i'm misunderstanding or things have changed since that SO post / that document was put up

Alex Miller (Clojure team)21:06:10

much of the stuff on the lazy page is in comparison to the "streams" idea that was under consideration

Alex Miller (Clojure team)21:06:33

and/or vs the prior version of laziness before the current version