Fork me on GitHub
#clojure
<
2018-06-13
>
noisesmith00:06:57

and so yeah - 6 is the max number and that's 2 more than my processor count, it's doing what's documented

seancorfield00:06:50

Right, I wasn't trying to prevent chunking. I should have pasted the earlier example from my REPL session to be less confusing.

seancorfield00:06:34

And with (take 100 (iterate inc 0)) I get up to 10 in-flight -- 2 + number of processors

👍 4
seancorfield00:06:41

So you get better "parallel throughput" if you operate on a chunked sequence...

noisesmith00:06:50

seems like it would be more useful to agressively dechunk inputs and make the n an explicit parameter

noisesmith00:06:55

(also simpler)

Jeremy00:06:59

Hi everyone. Has anyone here tried to use Cursive with reader literals within the ns form? I'm trying to load some libraries for clj and some for cljs, but I don't think Cursive understands it.

cfleming00:06:05

@clojure475 What’s the issue you’re seeing?

Jeremy00:06:44

It doesn't seem complete/understand items from the imported namespaces

cfleming00:06:11

Can you show the ns form you’re using?

Jeremy00:06:55

Sure just a sec

Jeremy00:06:53

(ns common.db
  (:require #?@(:clj [[clojure.spec.alpha :as s
                       java-time :as time]] ; `compare` works
                :cljs [[cljs.spec.alpha :as s]
                       [cljs-time.core :as time]
                       [cljs-time.extend] ; allows `compare` of `goog.date.Date`s
                       [goog.date]])

            [common.fsm :as fsm]
            [client.routes :as routes]))

Jeremy00:06:22

so when I try to type time/.... it offers cljs-time stuff, but not java-time

cfleming00:06:15

[clojure.spec.alpha :as s
 java-time :as time]

Jeremy00:06:19

oops it's actually offering *clj*-time, and neither cljs-time nor java-time

cfleming00:06:54

That bit there is incorrect, you have two require clauses in a single vector.

cfleming00:06:36

That said, it’s entirely possible that Cursive might get confused by this, CLJC is hard 😞

cfleming00:06:42

Let me know if it’s still not working.

Jeremy00:06:14

yeah that didn't fix it

Jeremy00:06:50

it's like it can't see the require at all

Jeremy00:06:58

(the bit within the reader literal)

cfleming00:06:50

Could you file an issue for that, and I’ll look at it? https://github.com/cursive-ide/cursive/issues

Jeremy00:06:27

thank you!

cfleming00:06:38

No worries, thanks for the report!

Jeremy00:06:57

I'm not quite at the skill level to help fix it yet sorry!

Jeremy00:06:02

will file report

skrat10:06:06

how can I write a parametrized (or generic spec)? say I have (s/def ::foo string?) (s/def ::items (s/coll-of ::foo) (s/def ::bar (s/keys :req-un [::items]) but the ::bar is so common in my codebase that I want a function that takes a predicate (`string?`) or a spec, and returns the new ::bar spec, that specifies a map spec with :items key, which should contain a collection of x (predicate or spec)

skrat10:06:45

this is stupidly simple to do with Schema

lmergen11:06:23

@skrat i think the answer to what you want is a multi-spec

lmergen11:06:22

you can basically parameterize the spec using (s/merge (s/keys ...) (s/multi-spec ...)), where you have a set of base keys and then merge into it another spec, which is dispatched based on for example a key's value

lmergen11:06:10

i've had quite some success with this with, for example, writing specs for a ::job, and then adding additional specs based on the job's type

skrat11:06:37

@lmergen interesting, but doesn't this presume I have a special field that indicates the "type" in that entity? because I don't have it and I don't need it there for any other purpose

lmergen11:06:41

so basically what you're saying is that you want the spec of ::bar to mean different things, depending on the context it's called in ?

lmergen11:06:14

(one time you want to use a string? predicate, another time something different -- let's call it a 'dynamic' spec)

skrat11:06:14

I don't need to call it anything or even be in the registry, I just need to use it for validation

skrat11:06:40

but that's optional, I could be doing that. the problem is, again, that spec is macro driven instead of data driven, so generating, or parametrizing specs is difficult

lmergen11:06:03

so multi-specs are one way to do that

lmergen11:06:23

let me think, because i think what you're really looking for is a function that returns a spec without registering it

skrat11:06:50

I know about spec-tools and data spec, that would solve the problem, but I'm trying to avoid it as an extra dependency

skrat11:06:18

indeed, that's what I want, a function that returns a map-spec

skrat11:06:07

maybe I should just use a macro for that

lmergen11:06:55

i think a macro will be your shortest path to victory here

matan15:06:56

How can we connect to a running program with a REPL? I saw some posts somewhere about exploring the state of a production app from a REPL, but I am unaware whether this can actually be accomplished with any clojure application, or does it actually prerequisite the application being started in a very certain way (or even requiring the application to include code that explicitly arranges this ahead of time).

matan15:06:19

Can anyone clarify about this?

seancorfield15:06:29

@matan there are a couple of ways: either the Clojure program can start an nREPL server and then you can connect to it with any nREPL client; or you can provide JVM options to the program at start up so it starts a Socket Server REPL and then you can use telnet or unravel etc to connect to it.

seancorfield15:06:27

We run a REPL inside several of our production processes for this purpose.

seancorfield15:06:14

(didn't I already answer this question from you in #beginners ?)

seancorfield15:06:10

@matan It's "just" a REPL. You can do whatever you want in it. There's no difference between a program you start from the REPL and a REPL you start from a program really.

matan15:06:55

I might find it hard to imagine how I’d use such a REPL running orthogonally to my application. Obviously it has access to the same namespaces, but spinning up extra copies of them is one thing, connecting to up and running atoms and agents quite another

noisesmith17:06:58

vars in namespaces are globals

noisesmith17:06:14

it's no different in a distributed repl than a normal one

seancorfield15:06:44

I'm not sure what you mean.

seancorfield15:06:40

The process in a REPL is exactly the same as how Clojure runs code anywhere: require a namespace, call a function...

seancorfield15:06:32

What do you mean by "spinning up extra copies of them"? Namespaces are singletons.

matan15:06:27

I’ll add a concrete example once in front of a desktop... but basically if I call up a namespace into the REPL, will it redefine all the vars in a separate “copy” that only my REPL is seeing, or will it just grant access to the already initialized copies of them running by the application?

seancorfield15:06:15

Namespaces are singletons. There is no "separate copy".

seancorfield15:06:16

If you require a namespace that is already loaded (and therefore compiled), it is a no-op. Well, perhaps beyond adding the alias to your current namespace.

seancorfield15:06:50

@carr0t You can start multiple REPLs.

seancorfield15:06:09

We're talking about connecting to a REPL running inside a program.

danm15:06:18

Aah, apologies

imetallica16:06:06

Folks, a doubt about spec: I’m trying to validate inputs and outputs of my functions using :pre and :post conditions, but I’m seeing that there is spec/fdef. What’s the difference between using one over another?

seancorfield16:06:56

:pre and :post are tied into the assertion machinery of Clojure (and can therefore be turned off globally). spec/fdef defines specs for arguments, return values, and the relation between -- and the arg checking is turned on/off via instrument and the return value/relation checking is done via check (both in clojure.spec.test.alpha).

seancorfield16:06:26

You can use specs in :pre / :post if you want (calling s/valid? on a spec and a value, for example).

imetallica16:06:14

So, to validate these specs in production what would be the recommended way of doing things?

seancorfield16:06:35

If you want to validate function arguments (in production), write explicit code in the function that calls s/valid? or s/conform.

seancorfield16:06:52

The assumption is that you use spec mostly around system boundaries. So you might use spec for validating/conforming incoming API or HTTP parameters, or around data you get from a third party service.

seancorfield16:06:16

And those tend to be places where you want the check to "always happen".

imetallica16:06:41

So is not recommended to turn on the assertion machinery in production?

seancorfield16:06:16

Depends who you talk to. I personally think that if you're going to use assertions at all, you should leave them on in production as well.

seancorfield16:06:24

But I generally do not like assertions.

seancorfield16:06:22

They throw Error rather than Exception which means you need to catch Throwable if you want to deal with them in your program.

imetallica16:06:50

I see, well, the problem is when they throw and return http 500 error on you

seancorfield16:06:05

A program can prevent that by catching Throwable and responding with something more friendly -- but if you're using assertions for "this can never happen" checks, then you probably want the program to crash and burn at that point.

👍 4
seancorfield16:06:59

Whereas data coming in from APIs, HTTP parameters, 3rd party services, perhaps even your legacy database... should all be "expected" to be invalid so the check should be part of your normal program execution logic.

imetallica16:06:26

Well, this is a new service I’m writing which will talk with an external service provider API. But the discussion was quite enlightening. Thanks! 🙂

seancorfield16:06:12

While you're developing, it can be very handy to have function specs enabled (via instrument) and to check the behavior of key (pure) functions. But instrumentation can be quite a performance overhead so you may well not want it on in production.

seancorfield16:06:57

Like most things, it's a trade off. For example, clojure.java.jdbc has specs for nearly all functions but they are optional, and running database access code with all of them instrumented is substantially slower -- but they can be a help during development to make sure your calls into that library are valid.

justinlee16:06:02

I need to take the first n elements of a collection where the sum of those elements is no greater than x. Can anybody think of a nice way of doing that? I was looking at medley’s take-upto, but that can’t carry state about the prior elements. The predicate needs to have reduce like flavor. I was thinking about maybe creating a lazy sequence of the sums and feeding that into take-upto but I couldn’t work out how.

justinlee16:06:38

(for the record, I have this working just fine with loop/recur.)

seancorfield16:06:49

@lee.justin.m You could always use reduce and stop with a reduced value when you have the appropriate number of elements.

justinlee16:06:15

@seancorfield oh cool i didn’t know about reduced. i’m not sure how that works but great!

justinlee16:06:48

would be nice if the reduce docstring mentioned that 🙂

seancorfield16:06:41

I suspect that when reduced and reduced? were added in Clojure 1.5, no one thought to update the docstring for reduce (which had been around since 1.0). It took up until Clojure 1.7 for the bugs to be shaken out so that all reducing functions respected reduced.

lsund16:06:11

Hello everyone, for those of you who use emacs: How do you automatically format your clojure code? cider-format-defun seems to not work and cider-format-buffer changes the cursor position. Ideas?

Mike Hay17:06:38

If you are just talking about indentation, C-x h (to highlight all) then tab.

lsund17:06:35

That does not appear to work for me...

Mike Hay17:06:34

Hmm. You also can try C-M-q on the highlighted area.

johanatan17:06:25

do people think it is appropriate for the clojure executable to print "Downloading: ..." messages to stderr? is this by design or a bug?

Alex Miller (Clojure team)18:06:01

you are welcome to criticize the design :)

johanatan18:06:13

can we at least count on the status code returned from the executable to be non-zero if an actual error occurred?

johanatan18:06:46

[i.e., i had some code that was checking for the combination of an empty stderr and a zero status code to indicate success]

johanatan18:06:10

i could relax that check to be just the latter if stderr is used this way intentionally]

Alex Miller (Clojure team)18:06:11

if there is a failure in building the classpath, you will get status code 1

johanatan18:06:03

[it's a bit of an abuse of stderr but i can understand why it was done this way (since there really isn't any other stream to send it to and we want the actual contents of stdout to be the literal classpath and nothing more)]

Alex Miller (Clojure team)18:06:45

I don’t think stdout is the classpath - we write all that to a cache file

johanatan18:06:15

hmm, that's not what i'm seeing. 2>/dev/null suppresses the "Downloading ..." messages but the classpath is still printed

Alex Miller (Clojure team)18:06:31

or do you mean specifically with something like clojure -Spath?

johanatan18:06:45

ah, yea clojure -Srepro -C:dev -R:lumo-compile -Spath 2>/dev/null is my invocation

Alex Miller (Clojure team)18:06:03

yes, the errors were separated for exactly this reason

johanatan18:06:13

makes sense. thx!

nickmbailey19:06:35

so we use plumatic/schema to coerce json back and forth from our internal schema, which works awesome for all our backend clojure services. unfortunately our frontend is javascript instead of clojurescript. wondering if anyone has found any good solutions for coercing json into more structured schemas in javascript

nickmbailey15:06:02

thanks for the lead! does that let you define custom coercions like date strings -> moment.js objects?

nickmbailey16:06:36

awesome thanks, looks like there are some extensions already for that

nickmbailey16:06:25

so i see how you can use this to take a json parsed object and convert it to a joi schema, but i don’t see how you might do the reverse. convert a schema to json

nickmbailey16:06:53

do you use it for that as well?

nenadalm17:06:07

no. I am using it just json->js (validation + coercion).

nickmbailey18:06:19

thanks again :thumbsup:

nickmbailey19:06:03

besides rewriting the frontend in clojurescript haha

4
hiredman19:06:11

have you seen json-schema?

nickmbailey19:06:32

yeah from what i could tell it’s specific to json but language agnostic. i’m more looking for something javascript specific that can for example coerce datetime strings into moment.js objects automatically

nickmbailey19:06:03

clojure has spoiled me

hiredman19:06:51

there are libraries that use json-schema to drive coercion

nickmbailey19:06:09

i guess thats what i’m having trouble finding then heh

hiredman19:06:57

I am not sure that can do custom coercion though

justinlee19:06:09

you could probably use shadow-cljs to create a library that uses plumatic/schema and then call that from javascript

justinlee19:06:46

it may not be svelte or quick, but it would work

nickmbailey19:06:58

yeah thats not a terrible idea actually

nickmbailey19:06:11

at the very least i would find it interesting to try in my spare time

justinlee19:06:11

allows you to reuse the work you’ve already done

justinlee19:06:07

the other thing that’s cool about that idea is that it allows you to slowly eat the javascript front end incrementally

lilactown20:06:58

Wrap it in a nice API and put it on npm 😉