Fork me on GitHub
#clojure
<
2017-08-07
>
qqq00:08:14

macros -- for () and [], the meta field gives us line/column; we do not get it for int/string/symbols - is there any compiler option which also allows macros to see th emeta info for symbols ?

the2bears03:08:59

Hmmm... no videos yet for EuroClojure

henrik06:08:17

I have here a ridiculous function I made for generatoring random dates:

(def simple-formatter (f/formatter "yyyy-MM-dd"))

(defn generate-simple-date []
  (f/unparse simple-formatter (c/from-long (math/round (* (rand) 9999999999999)))))
(it uses clj-time and math.numeric-tower) How do I turn it into a generator for use with clojure.spec?

misha08:08:14

@henrik kappa

(clojure.spec.gen.alpha/generate
  (clojure.spec.gen.alpha/gen-for-pred inst?))

;;=> #inst"1970-01-01T00:00:00.024-00:00"

lxsameer09:08:10

hey folks, I want to give a talk on a conference about clojure and want to encourage people to use it. do you have any reading or watching material to help me with the talk

oliy09:08:53

Anyone know if there's a slack channel where the Walmart devs hang out?

hmaurer10:08:30

@oliy there are some on #graphql chatting about Lacinia

misha13:08:36

if I have a "pure" lib, which is to be used with mount/component/etc. how do I arrange using dynamic binding(s) in my implementation? Is there a way to "detach" dynamic binding var from the top level of the file?

misha13:08:38

There is an atom, and an update function f, which can be called outside and inside of the add-watch function. If it is called inside of add-watchfunction, I want it to behave differently, than if called outside. More concrete example: make same function put event on one queue inside of atom, if event is external, and put event inside another queue, if event is emitted by event handler. Since both call sites will be implemented by user, I'd like to exclude possibility of user error, e.g. (emit-external!) inside handler.

misha13:08:18

which, on the second though, is sort of baby-sitting much, eh?

rickmoynihan14:08:33

I’m using clojure.test and one of my tests passes the value of *ns* in to a function (I expect it to resolve to the unit test namespace). This works fine at a REPL, however one of my namespaces does this and *ns* resolves as user not the test namespace when running lein test. Has anyone run into this problem/bug?

rickmoynihan14:08:17

my test namespace is pretty standard… it has a typical ns form at the top form - nothing fancy.

rickmoynihan14:08:37

I looks like it might be a bug in either clojure (1.9) or leiningen.

rickmoynihan14:08:31

I have some other namespaces that use the same pattern, and *ns* is bound correctly even when run via lein test

chouser15:08:36

@rickmoynihan ns is only certain to be defined the way you expect during compile- and macro-expand-time, not at runtime. So if you want to use its value at runtime, you might need to write and use a macro to capture its value and put it somewhere you can later find it at runtime.

rickmoynihan15:08:59

@chouser: interesting… can you explain why, and where this is documented?

rickmoynihan15:08:13

that’s very much not what I was expecting

chouser15:08:54

@misha I don't think I understand, but it sounds interesting. Do you already have a way to determine if code is being called by add-watch vs not?

chouser15:08:33

@rickmoynihan Well, anyone can set ns whenever they want, so its holistic behavior isn't (and probably can't be) documented. However, generally people leave it alone except by calling 'ns' and 'in-ns', which are documented to set ns. The rest of what I describe falls logically, if somewhat cryptically, from that.

chouser15:08:02

ugh, sorry, my asterisks are disappearing because I don't know how to slack.

rickmoynihan15:08:35

ok, yes that makes a lot of sense actually

rickmoynihan15:08:23

basically it’s a global set at read-time… if another namespace is evaluated it will stomp it for the duration of that namespaces evaluation

chouser15:08:52

Yeah, I think you've got it.

rickmoynihan15:08:28

Being thread local will prevent threads stomping on each other if namespaces are loaded in parallel; but as you say run time is after the fact.

rickmoynihan15:08:37

so all bets are off.

chouser15:08:48

so it's pretty safe to rely on the ns value at macro-expand time, because you generally do your (ns ...) at the top and then don't use it or in-ns again.

chouser15:08:02

yep, that's right.

rickmoynihan15:08:19

i’ll need to wrap my function in a macro; a bit of a PITA but no big deal.

chouser15:08:39

you shouldn't need a macro for a one-off. Just (def my-ns *ns*)

chouser15:08:35

You should only need a macro if you want different namespaces to be able to call your common thing, and that common thing needs to know the ns of the caller.

rickmoynihan15:08:47

yeah, I had thought of that… but it’s not a one off; it’s a pattern my tests are using to find some sidecar files based on the namespace name… I’d rather users had to just use the one function macro, rather than remember to def the ns-name.

chouser15:08:05

Yep, makes sense.

rickmoynihan15:08:17

yeah you kinda repeated me - I do want that 🙂

chouser15:08:05

Ah yes. I forgot your original question in my excitement over the behavior of *ns* 😛

rickmoynihan15:08:53

Anyway, thanks for your help… I should’ve realised what was happening - not sure why I thought this would ever work… I guess the fact it works at a REPL threw me.

rickmoynihan15:08:22

I clearly imagined some magic where there was none

misha15:08:57

@chouser yes, sort of. Lib api: 1) creates watch for you, and 2) provides slots for event handlers (think re-frame, I guess). So I "know" when an event is emitted by event handler (if handler happens to emit event), because I can call handler within a binding. (if the same handler is called outside of lib's event-loop – it will lack binding around it, and can be flagged as external emitter (or rather "not flagged as internal one")

misha15:08:51

if I'll give user 2 fns: emit-external, and emit-internal – using emit-internal outside of event-loop might lead to business-state inconsistency, unless I throw based on something

misha15:08:11

initially, I wanted single emit function, because 2 event queues is an implementation detail, which user should not really think/know/care about

leonoel15:08:02

@misha you should not rely on dynamic vars to implement this

misha15:08:03

so, say i went with bindings. Lib would look like: 1. (make-state) giving you atom with a state, with added event-loop watcher. You can have many of those. 2. global dynamic var {:state-atom-id false}, which would be updated with binding depending on which state's event-loop executes its handlers

misha15:08:21

and (2) might not even work, did not try it yet opieop

misha15:08:35

@leonoel can you suggest something instead? I don't like the whole ^:dynamic idea from the get go, but not sure what are the alternatives

leonoel15:08:12

for a similar problem, I used java.lang.ThreadLocal

misha15:08:07

1) I write it for cljs in the first place, but cljc is the end goal. 2) I did not really thought through how it should behave in multithreaded env (although FSMs have some concurrent things going on)

leonoel15:08:07

the problem is, if you rely on a dynamic var to test whether you're inside the event loop, suppose that a handler makes a call to future, because of binding conveyance the future will believe it's still in the event loop

misha15:08:35

that's true harold

leonoel15:08:52

if I remember correctly there's no binding conveyance in cljs ?

misha15:08:26

almost did not use bindings at all, can neither confirm nor deny : )

leonoel15:08:19

well I have no solution to suggest that would work out-of-the-box in both clj & cljs

misha15:08:06

what would work in clojure?

leonoel15:08:06

dynamic would work if there were'nt this weird behavior

misha15:08:01

I can't control handlers implementation, and can't/don't want to force those to be pure either

leonoel15:08:24

a threadlocal would work, declare it globally and make your call to the handler inside a try / finally to make sure the flag is reset at the end

leonoel15:08:44

just like binding is implemented, that is

leonoel15:08:20

then you poll the threadlocal to check if you're inside the loop

misha15:08:28

so if I allow i/o - emitted event might be one of side effects. And I need to prioritize events emitted from handlers, if any

misha15:08:33

I'll need to read about thread locals more, thank you

chouser15:08:38

How is Thread/local better than dynamic vars?

leonoel15:08:42

I'm not saying that threadlocal is better, I say dynamic vars have binding conveyance, which in this particular case makes them inappropriate

misha15:08:38

I always can put "don't do that" clause in read me opieop

chouser15:08:06

@misha Would it be good to provide all three? emit-external and emit-internal for users who are doing crazy or unexpected things with threads, and emit that tries to do the right thing based on the sensed calling context? If so, then would probably also be helpful to give the user access to the sense function, like internal? or some such.

chouser15:08:52

@leonoel Ah, I understand.

misha15:08:04

@chouser it would, but that requires solving emit : )

chouser16:08:19

Kinda weird -- I've never seen a situation where dynamic binding is almost right but you don't want conveyance.

misha16:08:55

the actual problem I am solving: I need to prioritize events emitted from event handlers, to implement: "run to completion" UML FSM requirement

misha16:08:25

I am very inclined to proceed with dynamics and just add a "don't" clause. But it might legitimately limit some use cases

chouser16:08:00

Not a bad place to start, I think, if you have the machinery behind a clean API.

misha16:08:18

or even with just emit-internal and emit-external, but that leaks implementation detail

chouser16:08:42

On the other hand, ThreadLocal is pretty easy: (def x (ThreadLocal.)) (.get x) (.set x 42)

misha16:08:46

did not use it at all, so I can't even assess it. Need to read about it some more

misha16:08:49

the role of dynamic was to hold a map of FSM ids to in-loop? flags.

misha16:08:17

and it should be visible from any thread, right? with thread local, there would be multiple projections of ThreadLocal's value, one per thread, right? and since thread can do 1 thing at a time, it cannot emit event both inside and outside the loop at the same time, right?

misha16:08:09

so all sync emit calls from within a loop would see "in a loop", and all async emit calls from within a loop would see "not in the loop".

misha16:08:11

what would be cljs implementation of this flag?

leonoel16:08:50

I would say, any mutable container such as atom or volatile! will behave like a threadlocal

noisesmith16:08:47

@misha my take on this stuff is that whenever possible a library should provide a pure function that takes all relevent data as args (including a params map full of stuff specific to that lib, if needed) - that’s the only thing that easily generalizes to dynamic binding, atoms, all the system state management libraries, etc. etc.

noisesmith16:08:34

it’s easy to build all of those things given a pure function, it’s not easy to do any of the other conversions (dynamic binding to global atom, global atom to system management lib, etc. are all hard things to do right)

misha16:08:22

@noisesmith I understand that, and can do pure implementation for everything but this emit use case.

misha16:08:25

and the only 100% pure solution I have so far – is 2 separate functions: emit-internal and emit-external

noisesmith16:08:03

be aware that if you use threadlocals you are ensuring that your lib can’t be used inside go blocks

noisesmith16:08:20

at least, not without a lot of pain and complication

misha16:08:52

but, if someone will use emit-internal either outside of the loop, or inside the loop in future - it will mess up FSMs semantics, so I need to guard from that somehow

misha16:08:25

@noisesmith ouch, core.async integration was next on the todo list harold

noisesmith16:08:00

what about using core.async to manage event ordering - reifying the emits as a series of events coming from multiple channels, so that a single block can ensure proper ordering

chouser16:08:02

@misha Would it be too ridiculous to have the user's functions return the things to emit, rather than calling emit as a side-effect? I think you ruled it out above for the general case, but even providing it as an option can be helpfully clarifying.

noisesmith16:08:34

I mean, it would be an option to make a “core async wrapper” that always launches a fresh thread and establishes the correct bindings and ensures your code never runs on a thread owned by go… but that feels a bit hairy too

misha16:08:06

I think anyone who'd care to "return what to emit" would use emit-internal/external just fine. or maybe I did not quite understand the solution

noisesmith16:08:16

if you use something other than threads to carry context (and explicitly carry it instead of implicitly) this stuff becomes a lot easier

noisesmith16:08:30

it’s reasonable to make an api that looks like a side effect to a client, but really returns data to your own managing context that does the real side effects

misha16:08:54

I thought that's what I'm trying to do here :) basically I need to know whether a function was called inside or outside the loop

chouser16:08:34

@noisesmith The user still has to be careful to return the thing in that case, right? An extra (prn ...) in the tail position ruins everything?

noisesmith16:08:00

well, what matters is what they provide to that api

misha16:08:10

right now, I keep FSM in atom, and expecting user to pass that atom as an argument to (emit) this way same function can be called inside/outside the loop, but this way I cannot put any context in it to identify the caller

noisesmith16:08:16

maybe my picture of this is wonky

noisesmith16:08:36

@misha that way the client is in charge of ordering

noisesmith16:08:54

doesn’t this get easier if there’s another layer that can fix the ordering?

misha16:08:11

probably, but I don't see it yet

noisesmith16:08:21

if the client ultimately decides on the ordering, all you can say is “you did your emit in the wrong order, it’s a problem in your code”

noisesmith16:08:16

but if you have some indirection between a client saying “this is data to emit” and the actual emission process, there can be something that knows “I have to wait for x before y can come out” or whatever

misha16:08:47

during event handling (FSM transition between states), there might be 2 event sources: outside (usual, in-click, on-response, etc.), and inside (events emitted as side effect of FSM transition/entry/exit behaviors). To satisfy "run until completion" requirement, I need to ignore any outside events while transition is in progress, but prioritize and execute inside events.

misha16:08:30

I implemented it with 2 queues of events. When machine is in stable state (idle), only outside q accepts events, but when there is transition in progress - second q might receive some events. And the only way I can tell where to put an event, is by knowing was it emitted by handler from inside the event-loop (calling same handler outside the loop, even if at the same time, but from different call site - has to put event in outside q, where it'll wait for idle machine state).

noisesmith16:08:44

right, that’s easy with a couple of queues and an agent or go block between the user and the atom

misha16:08:51

I want to avoid baking in core.async at this point, will try to wrap it later as a separate integration, but did not thought about it yet.

chouser16:08:51

So what if the only way to get events into the high-priority queue were for user event handler functions to return data that meant "here are events to queue". Then the only public 'emit' side-effect function would always add to the low-priority queue.

noisesmith16:08:12

that sounds about right, yes

misha16:08:47

@chouser easy to make a mistake, want to lift as much pressure from user as possible

noisesmith16:08:09

only the internals of the lib need to access the higher priority queue (I have something just like this in the code I should be working on right now…)

chouser16:08:15

what kind of mistake would be easy?

chouser16:08:30

(I don't doubt you, just trying to understand)

misha16:08:41

not every event handler (actually, it is transition/entry/exit behavior) will emit event

misha16:08:15

and i want those to be arbitrary functions, with only constrain being args list

misha16:08:55

actually, wait, those behaviors have to return db (state) I guess. Hm, I need to think about that. if there is inherent constrain on return value - then yes, emitting priority events will be done through separate pure fn only.

chrishacker19:08:17

Does anyone have a suggestion or example of building a multipart form submission in clojure? I've been trying with http-kit unsuccessfully. I need the first part to be json and the next part to be a file. To be clear, this is effectively an upload POST call, but server to server.

jeff.terrell19:08:52

I think the first thing I'd do is look for is Ring middleware to handle multipart form submissions.

chouser19:08:18

@chrishacker Oh, you mean acting as a client?

chrishacker19:08:40

Thanks jeff, but I'm sending, not receiving.

chrishacker19:08:12

I already am using ring to receive file uploads.

chrishacker19:08:36

At a later point in the lifecycle of the file, I need to submit it to a 3rd party and they require a multipart form.

chouser19:08:30

looks like http-kit has code meant to support it

chrishacker19:08:43

meant to is the right phrase

chrishacker19:08:50

it builds a multipart post that only has one type allowed. I need a more complex body, though pretty standard

chrishacker19:08:30

Authorization: Bearer OvwkFQIqMsi5vTGq+tTAAAAAxKM=
Accept: application/json
Content-Type: multipart/form-data; boundary=01ead4a5-7a67-4703-ad02-589886e00923

--01ead4a5-7a67-4703-ad02-589886e00923
Content-Type: application/json
Content-Disposition: form-data
{
  "status":"sent",
  ....
}
--01ead4a5-7a67-4703-ad02-589886e00923
Content-Type: application/pdf
Content-Disposition: file; filename="test1.pdf";documentid=1

[PDF binary document content not shown]

--01ead4a5-7a67-4703-ad02-589886e00923--

chouser19:08:44

If http-kit can't do it, I'd probably use apache libs with Java interop directly, and follow a stack overflow example. shrug

chrishacker19:08:14

yeah, was worth asking.

chouser19:08:08

That snippet you posted is what you want, or what you're getting from http-kit?

chrishacker19:08:27

It won't do the content in the first boundary set

chrishacker19:08:51

(or I'm missing something )

ghadi19:08:51

javax.mail is a horrible horrible library that I've successfully used to create non trivial MIME constructions. You may want to look at it.

chrishacker19:08:11

Thanks for sharing your pain 😉

ghadi19:08:47

create the mime body using that, then shove it into httpkit

chrishacker19:08:22

I'll take a look

chouser20:08:15

It's a little unusual to send form data in the first part as JSON rather than url-encoded, isn't it?

chrishacker20:08:11

Not my call - docusign API

qqq23:08:00

what graphics library is best to "blit" a 80x25 monospace font to screen? so my clojure data structure represents an 80x25 array of characters, the yare all monospaced, no ligatures / fancy font rendering; I need to blit them to screen to display. What java API will provide the fastest way for doing this?

noisesmith23:08:25

if you can use any kind of display, ncurses in a terminal might work pretty well (dependent on how good your user's terminal is of course)

tbaldridge23:08:39

"What java API will provide the fastest way for doing this?" Vertex buffers on the GPU. Upload the font to the GPU using a large bitmap, set the U/V of each vertex to match the corners of each letter in the bitmap.

tbaldridge23:08:00

Bonus points if you use vertex shaders to generate the UV coordinates via a vertex shader. So you upload text, the GPU does everything else.

tbaldridge23:08:16

Funny enough, that's about 20 lines of GLSL ^^