Fork me on GitHub
#clojure
<
2019-02-08
>
joshkh01:02:36

this might be a silly spec question, but is there a way to use a def'ed collection of keywords with, say, the :opt group in s/keys? i'm trying to define a map with a large list of optional keywords defined elsewhere:

(def some-test-keys [:test/a :test/b :test/c ... +100 more])
(s/def :test/labels (s/keys :opt some-test-keys))
CompilerException java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Symbol

seancorfield02:02:28

@joshkh Not with spec as it stands, but programmatic specs are coming in the next version (aka spec2).

joshkh05:02:19

ah clever! i did something similar to turn forms back into spec maps. it didn't cross my mind that i could use the same technique -- thanks for the suggestion. https://github.com/joshkh/specalog/blob/dev/src/specalog/jig.clj#L9

seancorfield02:02:05

What you can do is define the spec with the literal set of keys and then use s/form to get back the form of the spec and take it apart to get the list of keys in it. In other words, let the spec by the "System of Record" and extra your "collection of keywords" from it.

seancorfield02:02:13

user=> (s/def ::my-spec (s/keys :opt [::key1 ::key2 ::key3]))
:user/my-spec
user=> (let [[_ _ ks] (s/form (s/get-spec ::my-spec))] (println ks))
[:user/key1 :user/key2 :user/key3]
nil
user=> 
You can get fancier and check what you get back from s/form actually has clojure.spec.alpha/keys as the first element and :opt as the second element and/or handle :req/`:req-un`/etc as well.

fenton03:02:44

is there an equivalent to defonce like doonce I just want to execute some code only one time, not each time I reload my src file.

fenton03:02:20

I'm currently doing defonce ignore (blah....

seancorfield03:02:35

Having code executing when a namespace is loaded is generally a bad idea (since it will run if you compile the ns, for example when building an uberjar with Leiningen or Boot).

dominicm06:02:16

A bit out of context, but in clojurescript this is normal in order to activate your application.

seancorfield03:02:02

If this is for dev work, you're probably better off putting it inside a (comment ..) form and manually evaluating it when you need to @fenton

fenton03:02:02

@seancorfield thats a great point, thanks sean! stinky code 😉

seancorfield03:02:14

Many of my source files end with a (comment ..) form that contains all sorts of expressions to do dev setup and some testing, including requires that are only needed for dev work, setup for Components, etc.

alex07:02:07

Hi all! Is it possible to define mutually recursive functions with metadata in a local scope? I want to define f and g, and attach metadata (using with-meta or something similar) such that each can see the other's metadata within its body.

alex07:02:40

(This has come up while implementing a fn-like macro that creates a function and attaches some metadata to it. I'd like these "fake fns" to be useable anywhere a real Clojure function can be made, which led me down the road of trying to create a letfn-like macro that attached metadata to all the functions declared in the letfn. But I ran into the issue of not being able to create a letfn that binds metadata to the functions it creates.)

alex07:02:22

I guess something like this should work:

(letfn 
  [(f-thunk [] 
     (with-meta (fn f [] (meta (g-thunk))) {:b 3})) 
   (g-thunk [] 
     (with-meta (fn [] (meta (f-thunk))) {:a 2}))] 
  (let [f (f-thunk) g (g-thunk)] 
    [(f) (g)]))

jaihindhreddy09:02:52

Apologies if this is noise, but is there any plan to add metadata to EDN?

kwladyka10:02:13

Ech I always forget this…. (re-matches #"(?i)<message>(.+)</message>" "<message>foo</message>baz<message>bar</message>") How to get foo and bar from here? I am trying half an hour 🙂

seegy10:02:49

Hi there! Is there anybody, who knows about a problem with slurp? I try to fetch some websites, that I can reach via browser, but slurpthrows an exception:

(slurp "")

FileNotFoundException   sun.net.www.protocol.http.HttpURLConnection.getInputStream0 (HttpURLConnection.java:1836)
Tested the same URL with curl and it works fine. Any ideas?

Chris10:02:02

Any more useful information in the stack?

seegy10:02:42

lein version
Leiningen 2.8.3 on Java 1.8.0_91 Java HotSpot(TM) 64-Bit Server VM
org.clojure/clojure “1.9.0”

Chris10:02:46

hmm, I get the same problem

Chris10:02:51

curl -i ... shows it's speaking HTTP/2, maybe something to do with that?

seegy11:02:35

:man-shrugging:

seegy11:02:45

That's why I'm asking

jumar18:02:16

Could you be more specific and given an example of url reproducing the problem?

phill10:02:21

@kwladyka re-seq . But the regex is not quite right - the .+ is too greedy and may consume the end tag

kwladyka10:02:59

sure, but the question is how right regex look 😉

phill10:02:45

The question makes it less greedy

kwladyka10:02:52

nope, it doesn’t help

phill10:02:14

Did you try it at the REPL? I did

kwladyka10:02:25

oh it works with re-seq

kwladyka10:02:43

I am after week 16h work everyday after breaking changes third party system update. My mind is almost turned off 🙂

phill10:02:57

Try it at the REPL and you'll see

leonoel12:02:16

definline is still labeled as experimental in the docs, but it seems hardly ever used. what is its official status ?

Alex Miller (Clojure team)13:02:18

Would like to deemphasize - primarily used in Clojures implementation

leonoel13:02:40

so better not rely on it ?

erwinrooijakkers12:02:36

@kwladyka another approach:

(require '[hickory.core :as hickory]
         '[hickory.select :as select])

(def parsed-doc
  (hickory/as-hickory (hickory/parse "<message>foo</message>baz<message>bar</message>")))

(mapcat :content (select/select (select/tag :message) parsed-doc))
;; => ("foo" "bar")

kwladyka12:02:02

Can it find all <errors><error><messsage>…..</message>… in string? Because in reality it is very complex XML structure.

kwladyka12:02:19

and this message can be everywhere

kwladyka12:02:29

so not sure it will be more efficient

erwinrooijakkers12:02:37

It is bad practice to use regex for parsing XML 😉 See this famous answer: https://stackoverflow.com/a/1732454/2609980

kwladyka12:02:25

Sure it is, but in many cases it just work as simple 1 line solution

kwladyka12:02:46

But I agree

erwinrooijakkers12:02:16

Sometimes it can be easier

erwinrooijakkers13:02:18

Alright let me know how it goes

kwladyka13:02:39

Unfortunately I don’t have time to try it now. But I wrote it in my notes to try later 🙂

witek14:02:39

I have a collection c which can be a list, a vector or a set. How do I remove an item from it while keeping the type of c?

Alex Miller (Clojure team)14:02:30

those are different kinds of things so there is no common operation to do that

Alex Miller (Clojure team)14:02:55

usually if I encounter a question like this, I step back and look at how I got to wanting to remove things from a non-indexed collection

witek14:02:33

@alexmiller So you mean, I should avoid having to remove individual items from lists or vectors? Just from sets?

Alex Miller (Clojure team)14:02:28

yes, generally I try to never remove anything from the middle of a list or vector

Alex Miller (Clojure team)14:02:45

sets are fine - they are designed for that

witek14:02:57

@alexmiller Thank you. I'm going to think about that...

vlaaad14:02:46

@witek if you are okay with O(n):

(let [xs [1 2 3] ;; or '(1 2 3) or #{1 2 3}
      el 1]
  (into (empty xs) (remove #(= % el)) xs))

noisesmith16:02:38

beware that into on a seq reverses order

kana15:02:39

How does Calva find test namespaces? I have ["src/a/b.cljc", a.b] and ["test/a/b_test.cljc", a.b-test], and :source-paths ["src", "test"], but Calva says "No tests found. 😱"

kwladyka15:02:36

#calva-dev try here

Jacob Haag17:02:53

Is there a more idiomatic way of doing an or check with multiple = checks involving the same variable?

(or (= panel-name :general-info) (= panel-name :permanent-address) (= panel-name :school-info))

wilkerlucio14:02:19

@U994BRXMF try (#{:general-info :permanent-address :school-info} panel-name)

dpsutton17:02:07

(#{:general-info :permanent-address :school-info} panel-name)

Jacob Haag17:02:58

Ah yes, I have used something like this before.

Lennart Buit19:02:01

May I ask, I was wondering why this works, is a set somehow also a function?

Lennart Buit19:02:33

I know it does work, but come to think about it I don’t know why :’)

markw19:02:36

Yes, just like map

Lennart Buit19:02:42

(I meant under the hood)

markw19:02:32

and I meant, similar to map, it implements IFn

Lennart Buit19:02:03

Right I see now, then we are on the same page. Thanks!

markw19:02:24

np...when I first came across this I was both puzzled and delighted

Lennart Buit19:02:30

Yeah I know it works for months now, but I never stopped an thought about why

Lennart Buit19:02:49

What a curious interface iFn, it has invokes for 0..20 arity plus 20 arity and varargs

markw20:02:18

That stuff is all performance related as far as I understand it

markw20:02:32

usually the interesting ones are at the bottom where it explicitly handles multi-arity

dpsutton20:02:05

there's some different between a form run in the repl, compiled in the repl and run normally too. invoke, run, etc

Lennart Buit20:02:09

Yeah I know that varargs are slow

dpsutton20:02:21

but it's so hazy for me that i'm almost certainly wrong but just a grain of truth there

dpsutton20:02:39

if you ask in #clojure i'm sure someone could point you towards that

dpsutton20:02:48

(if you're interested)

Lennart Buit20:02:06

Always interested in more knowledge, but not always at the expense of time of others ;)

Lennart Buit20:02:11

If that makes any sense

markw20:02:17

class clj_dict(dict): def call__(self,item,default): return self.get(item,default) f = clj_dict(a=1,b=2) print(f('c','not_here!'))

markw20:02:26

my python will never be the same 😛

Lennart Buit20:02:22

Same goes for equality, math ops, lookup, etc

Lennart Buit20:02:59

I’ve seen python code that repurposes ** for something clever

markw20:02:45

yeah it has pretty convenient metaprogramming with it's "magic" methods

markw20:02:57

nothing beats macros though

Lennart Buit20:02:03

Language design is such a fascinating subject

markw20:02:00

definitely... also quite the rabbit hole

markw20:02:12

i prefer being a spectator and letting the adults play

noisesmith17:02:24

contains? at the front if nil or false are one of your cases

noisesmith17:02:50

the set works because a set acts as a function returning an item if found in the set

dpsutton17:02:03

yeah. however it looks. but rather than or just do set membership check

dpsutton17:02:20

which is semantically what you are doing

noisesmith17:02:07

also, depending on where the or is:

(case panel-name
  (:general-info :permanent-address :school-info) (some-action))
- but this only works if your matches are compile time literals

noisesmith17:02:36

(the case would potentially replace if/cond here)

Alex Miller (Clojure team)18:02:06

(#{:general-info :permanent-address :school-info} panel-name)

Alex Miller (Clojure team)18:02:17

^^ enumerate things in a set and use set membership

Alex Miller (Clojure team)18:02:48

literally, is panel-name in this set of values

Alex Miller (Clojure team)18:02:17

if you want to improve readability you can make that more explicit: (contains? #{:general-info :permanent-address :school-info} panel-name)

Alex Miller (Clojure team)18:02:57

oh sorry, I didn’t see @dpsutton s reply above, sorry to duplicate

dpsutton18:02:23

i'm always pleased when my recommendation aligns with yours 🙂

Bravi19:02:09

hi everyone. are there any isomorphic web frameworks for clojure, except fulcro?

NoahTheDuke19:02:51

what's an isomorphic web framework?

Bravi19:02:40

basically a combo of server - side rendering and a single page application

Bravi19:02:08

where you can use the same exact component on client and server

NoahTheDuke19:02:59

ah cool, okay

john19:02:53

most cljs react wrappers can probably be retrofitted to work isomorphic style, with some elbow grease. But rum was able to do it out of the box when it first shipped (or pretty early)

Christian Johansen21:02:18

tooting my own horn here, but https://github.com/cjohansen/dumdom can render the same components server and client side on both the JVM (native, no JavaScript runtime) and Node

pepas22:02:39

I hope I'm not beating a dead horse, but I wanted to ask some conceptual questions about clojure startup time. Some things I understand: - The JVM startup time is a relatively small component - Clojure itself has to evaluate all of its internal code when you start it up, which is the bulk of the time - I've read that some lisp implementations are able to save an "image" of a system - Subsequent invocations can start quicker by simply loading the image - I think GraalVM supports a similar notion, but that might be only for compiled code (you lose eval?) - Clojurescript / Lumo, etc have a faster startup time (on platforms for which Node is available)

pepas22:02:13

My question was: are there any technical limitations which would make it impossible for clojure to eventually support this notion of saving to / starting from an "image"?

pepas22:02:29

i.e. is there anything about the JVM which would make that impossible?

pepas22:02:03

Startup times on machines I own, if anyone is curious 🙂

startup times:
* macbook pro 2015: clojure(1.9):         ~1.2 seconds
* macbook pro 2012: clojure(1.9):         ~1.2 seconds
* HP [email protected]:     clojure(1.8,openjdk): ~6.1 seconds
* raspi 3:          clojure(1.8,oracle):  ~16 seconds
* OLPC:             clojure(1.6,openjdk): ~20 seconds
* raspi zero w:     clojure(1.6,oracle):  ~81 seconds
* raspi zero w:     clojure(1.6,openjdk): ~97 seconds
* pogoplug:         clojure(1.6,openjdk): ~98 seconds
* NSLU2:            clojure(1.9,openjdk): ~284 seconds

noisesmith22:02:51

what are you starting in each of those cases? lein? the clojure command line tool? java -jar ... ?

pepas22:02:13

oh, sorry: time echo '(System/exit 0)' | clojure

pepas22:02:18

not sure how scientific that is

noisesmith22:02:25

so the clojure command line tool

noisesmith22:02:54

just make sure it's not the first run from that directory (cached deps list / downloads)

pepas22:02:57

(I should also state that I'm a clojure noob -- please point out any glaring mistakes I'm making)

pepas22:02:17

@noisesmith yeah, thanks, I did see a speedup on subsequent runs!

noisesmith22:02:32

there are a few things that can be sped up, where speeding them up removes desired dev-time features of the language (eg. redefinitions of vars) - some of them are provided by default for clojure.core but can be turned on for other namespaces

pepas22:02:57

oh interesting

pepas22:02:55

for the "image" idea, I'm curious how that is implemented on other lisps. literally dumping a copy of the stack / heap to disk? seems like that would be large

dpsutton22:02:00

direct linking?

👍 5
noisesmith22:02:58

@jasonpepas precisely, it's a space/speed tradeoff (java makes enough of those already that the size of allocations becomes a factor...)

👍 5
noisesmith22:02:34

yeah, that's the one

noisesmith22:02:37

@jasonpepas the jvm arg is -Dclojure.compiler.direct-linking=true

👍 5
pepas22:02:47

As of Clojure 1.8, the Clojure core library itself is compiled with direct linking. oh interesting, so I should definitely be comparing 1.8 to other versions

noisesmith22:02:23

I guess it doesn't apply to your test, as it defines no clojure code outside clojure.core

pepas22:02:28

might explain why the raspi zero w was way slower than I expected vs. the rasp 3 -- I was running 1.6 on one and 1.8 on the other

pepas22:02:09

@noisesmith oh, if I don't use clojure.core then it doesn't get evaled on startup?

noisesmith22:02:55

what I mean is that the flag I menion above doesn't do anything if you aren't compiling anything - I guess your one interop call gets compiled but that should be minor

pepas23:02:03

oh gotcha

noisesmith23:02:34

if you had a significant app to compile and run, I would expect that flag to make a difference

pepas23:02:00

One the conceptual things I seemed to gather was that porting clojure to a different platform doesn't address the startup issue, because it is mostly due to the need to evaluate a bunch of clojure code on startup. I grabbed a copy of clojure_py and observed similar results.

pepas23:02:19

(so i.e. porting to C wouldn't help anything, and might even be slower the the JVM)

pepas23:02:20

The thing which struck me was the difference between clojure startup times and some common-lisp implementation startup times. e.g. on my lowly NSLU2, gcl starts in 0.26 seconds. So it must not be evaluating much on startup.

pepas23:02:46

Anyway, just wanted to ask if there was any knowledge / current efforts around the idea of starting clojure from an "image" in a way similar to e.g. common-lisp (I think emacs does this as well?) (related: https://stackoverflow.com/a/7686350/7543271)

seancorfield23:02:24

While there is ongoing work to help reduce Clojure's startup time on the JVM, the idea of an "image" is a non-starter, in my opinion, since you're talking about the JVM itself really -- you'd need to be able to save and load JVM images for Clojure to be able to leverage that.

Alex Miller (Clojure team)00:02:27

AppCDS does this on the jvm and it does help a lot, but kind of requires you to know everything you’re going to load

pepas23:02:38

ah, gotcha

seancorfield23:02:31

Clojure just has a lot of .class files to load and initialize.

seancorfield23:02:04

It would require a fairly fundamental change to the (Clojure) compiler to produce something that loaded and initialized faster for clojure.core (and the handful of other namespaces that actually get pulled in for -main to run) -- and that's just not a concern for server-based software which is really Clojure's target.

pepas23:02:36

thanks for the perspective @seancorfield

darwin23:02:12

I would recommend watching this great talk from Gary Fredericks: https://www.youtube.com/watch?v=-Qm09YiUHTs - IMO there is no low hanging fruit (without introducing breaking changes)

pepas23:02:35

@darwin thanks I'll check it out!

seancorfield23:02:39

On a reasonably fast laptop/desktop, clojure is fast enough for scripting for most purposes tho', unless you're looking for sub-second response times...

seancorfield23:02:54

Gary's talk is excellent!

pepas23:02:28

yeah, I gather that most people are using a SLIME-style workflow anyway, so they infrequently see the startup time lag

seancorfield23:02:53

I guess a lot depends on which languages you're coming from... I've been on the JVM primarily for about twenty years so... 🙂

👍 5
seancorfield23:02:43

Aye, I start a REPL up and leave it running for days. In fact, I often have two or three REPLs running in different contexts, and switch my editor between them as needed.

5
👍 5
☝️ 5
seancorfield23:02:04

And "Welcome to Clojure!" @jasonpepas 🙂

seancorfield23:02:37

(just scrolled back and saw your comment that you're new-ish to the language)

seancorfield23:02:46

One difference between 1.8 and 1.9 is the addition of clojure.spec and core.specs so I suspect that's part of the slowdown from 1.8 to 1.9

Alex Miller (Clojure team)00:02:47

Correct. 1.8 had some cost from incidental loading from the socket server stuff

pepas23:02:07

ah interesting

pepas23:02:49

yeah even though I'm new, I've been watching rich's talks for years and they have influenced my thinking in other languages. my goal for 2019 is to get serious with clojure

seancorfield23:02:20

Locally, it looks like startup times were pretty consistent from 1.2 up to 1.7 (about 1.15 - 1.2s on my desktop), then dropped to just under 1s with 1.8, and went back up to about 1.1-1.2 with 1.9.

pepas23:02:01

ah, so that reflects the change to direct linking in core and then the introduction of spec

seancorfield23:02:43

And 1.10 and 1.11 (master) seem about the same as 1.9 (again, consistent).

pepas23:02:52

oh thanks!

seancorfield23:02:50

You can shave some startup time off for clojure with -J'-Xverify:none'

seancorfield23:02:05

(per that wiki page -- but see the caveats around that)

seancorfield23:02:46

This is consistently 0.9s for me

time echo '(System/exit 0)' | clojure -A:1.10 -J'-Xverify:none' -J-client -J-XX:+TieredCompilation -J-XX:TieredStopAtLevel=1

👍 5
hmaurer00:02:13

what machine are you on?

seancorfield01:02:19

A late-2012 27" iMac with 16GB RAM and a 3.4GHz Core i7 with SSD drives.