Fork me on GitHub
#clojure
<
2017-04-29
>
bradford00:04:35

Does ring's multipart-params middleware turn binary application/octet-stream uploads into an ASCII string? How do I turn that into something I can serve as a file in an HTTP response? The byte-streams library can't seem to do it with to-input-stream. (like with a jpg image)

bradford00:04:23

I've got

(defn stream->bytes [is]
  (loop [b (.read is) accum []]
    (if (< b 0)
      accum
      (recur (.read is) (conj accum b))))) 

(defn body[resp] 
                   { :body    (with-open [is (byte-streams/to-input-stream (:upload (:multipart-params resp)))]
                               (stream->bytes is)) }
                    ) 

bradford00:04:39

But that produces a org.eclipse.jetty.io.EofException: Closed

bradford00:04:53

And {:body (byte-streams/to-input-stream (.getBytes (:upload (:multipart-params resp)) StandardCharsets/US_ASCII))} doesn't seem to render properly either

noisesmith00:04:05

@bradford what is the type of (:upload (:multipart-params resp)) ?

noisesmith00:04:29

I would assume would be a better function there as compared to byte-streams/to-input-stream - just going on the fact that it takes a char encoding arg if nothing else

xiongtx04:04:05

Why does clojure.lang.RT/canSeq need to check both ISeq and Seqable when ISeq <- IPersistentCollection <- Seqable?

static public boolean canSeq(Object coll){
    return coll instanceof ISeq
            || coll instanceof Seqable
            || coll == null
            || coll instanceof Iterable
            || coll.getClass().isArray()
            || coll instanceof CharSequence
            || coll instanceof Map;
}

qqq05:04:40

( (+ x 2) (+ y 3)) would be like (x+2) (y+3) Is there a structured way to convert sexp <-> math notation ?

octahedrion05:04:16

@qqq yes, and it's a fun puzzzle

rauh05:04:24

@qqq there's a few. Search for clojure infix : Example: https://github.com/rm-hull/infix ,

octahedrion05:04:27

in Rich's talk on Spec, https://vimeo.com/195711510, he mentions that vars are "fully reified" - what does he mean by that ?

rauh05:04:53

@octo221 reified => It's data, not just syntax as with many other programming languages. You can inspect them, assign them, pass them around etc.

octahedrion05:04:39

ah code-as-data

octahedrion06:04:15

then around 37:15 he says he would reduce the amount of reification in the var system - why ?

octahedrion06:04:18

does he mean that fns are too coupled to their implementations ?

octahedrion06:04:15

I guess it's about wanting more late-binding right ?

matan12:04:33

short of using low level monitors on objects, have we a way to synchronize side-effecting functions across threads?

leonoel12:04:59

agents and go blocks are side-effect friendly

matan12:04:04

thanks. looking for synchronous operation, agents won't do, but I will be looking at core.async (go blocks) for establishing something like actor semantics

leonoel12:04:12

yes they're both asynchronous constructs if you need synchronicity, monitors are the way to go

matan12:04:38

yes but kind of ugly in clojure, feels like a java singleton, unless I'm missing something

matan12:04:00

now I kind of forgot why actors were designed to re-try rather than lock in the first place

leonoel12:04:29

I don't get the singleton analogy

matan12:04:14

Right, irrelevant analogy. I'll stick to only clumsy, but hey not too much

matan12:04:23

Thanks for the tips! will manage from here

tbaldridge13:04:07

@octo221 right now vars do way too much. Var metadata holds: doc-strings, information about when the var was added, inlining information, linking information, arglist info, etc. Most of that stuff doesn't need to be there, but over time var metadata has just been the place where we dump everything that needs to be named.

tbaldridge13:04:28

that's why spec has its own registry, the var registry is polluted enough as it is.

tbaldridge13:04:32

@matan don't do side-effects in go blocks they have a limited thread pool. Use clojure.core.async/thread instead

leonoel13:04:28

@tbaldridge non-blocking side-effects are ok in go blocks right ?

octahedrion13:04:44

@tbaldridge ahh right - cheers!

tbaldridge13:04:46

@leonoel yeah, that's fine, and side-effects in general are fine (like a println) it's just that if the tread blocks for awhile it can cause a deadlock, but not a problem unless your sideffects take longer than 100ms or so

leonoel13:04:13

same problem with agents and monitors, actually

octahedrion13:04:31

@tbaldridge also the questioner referred to the var & spec registries as "databases"...I guess that's what they are in a way...one could add re-frame's event registry to that and others. Given that there's all these ways to manage naming of things, has there been much discussion about other ways to do it ? I remember there was once an experiment with immutable namespaces too

tbaldridge13:04:01

@leonoel not really, agents are given a separate thread per agent if you use send-off

leonoel13:04:54

yes, I mean you need to know if your call is blocking or not and choose the pool appropriately

matan13:04:47

I am trying to recap on atoms v.s. monitor locks (now that I'm using locking for a side-effecting database operation that is called from many threads). Is it correct that atoms can never run into deadlocks nor into very terrible livelocks?

matan13:04:00

And does an atom swap! slow down execution far less than a lock?

leonoel13:04:35

atoms are not suitable for side-effecting operations because they can be run several times when run concurrently

matan13:04:45

yes I know, I try to recap on the general (non-side-effecting case though)

leonoel13:04:00

atom use compare-and-swap which is generally faster than locking

leonoel13:04:13

and I wouldn't say they're guaranteed deadlock-free

noisesmith14:04:05

leonoel: if you are using swap! there's no code path with multiple swap! calls that leads to deadlock

leonoel14:04:29

let me think about this ...

noisesmith14:04:12

compare-and-set is proven to be a lock-free programming technique, and the inability to cause deadlocks is literally the definition of "lock free" in cs literature - and compare-and-set is the only coordination used by swap! assignment

leonoel14:04:37

thanks for the reference

leonoel14:04:50

is deadlock a term specific to mutexes or is it generic to any situation where on thread waits for something forever ?

noisesmith14:04:50

it's generic for any case where a thread never proceeds

leonoel15:04:55

ok got it, I was conflating deadlock / livelock / starvation, thanks for the reminder ๐Ÿ™‚

matan13:04:24

I've long forgotten why locking is more expensive though... might just take it as a given

noisesmith14:04:51

matan: it's more expensive because using a lock is a multi-step process that isn't free. It also easily leads to subtle bugs that cause deadlock which is infinitely expensive

leonoel14:04:10

anyway, it's just a rule of thumb, if there is high contention you'll need to benchmark

matan14:04:55

thanks for all the support. working with a MySQL auto-incrementing key in code that needs to be synchronized was kind of a pain ๐Ÿ˜“

matan14:04:15

a good old lock was the only solution

leonoel14:04:04

get-scenario-id-from-db is not thread safe ?

matan14:04:48

it is thread safe but runs a database transaction having get-or-set semantics

leonoel14:04:05

if it's thread safe, you don't need to lock

matan14:04:34

it is thread safe on the local JVM, but concurrency-unsafe for the system as a whole, if the database get-or-set is not synchronized

donaldball14:04:18

My experience is that if your database transaction needs support from its callers in order to maintain your isolation and consistency requirements, you probably need to rework the transaction. ๐Ÿ™‚

matan14:04:37

@donaldball you are probably right

leonoel14:04:38

the lock is local to JVM as well

matan14:04:47

@donaldball or rethink the schema ๐Ÿ˜ญ

matan14:04:53

well auto-incrementing keys come with their responsibilities

donaldball14:04:55

There is a #sql room which might be more appropriate for an exploration of that, but one place to start might be increasing your txn isolation level to SERIALIZABLE, or if youโ€™re doing an upsert use the mysql-specific syntax to do that in one statement

matan14:04:32

:thumbsup:

dthurn15:04:52

q: is it possible to dynamically load namespaces in clojure based on runtime state? I get ClassNotFoundException if I try to 'require' inside a function body

noisesmith16:04:04

@dthurn to compile code that isn't available yet, you need to use a dereference to lookup the values at runtime, this is typically done with resolve which returns a var (if it can be found)

noisesmith16:04:02

anyway, that works in a repl

dthurn16:04:10

looks like find-ns and ns-resolve is happening then

noisesmith16:04:12

resolve prevents the language from helping you

dthurn16:04:27

just trying to work around a library that does a bunch of annoying stuff at load time

noisesmith16:04:35

@dthurn ns-resolve says "look this symbol up as if we were currently in some other ns" - it's usually not what you want

noisesmith16:04:54

resolve is the simpler one

dthurn16:04:02

ok, got it

noisesmith16:04:55

eg if you have (ns foo.bar (:require [baz as b])) you can do (ns-resolve 'foo.bar 'b/f) to find baz/f

danp20:04:57

Hey all, I'm trying to set up Cloud9 (https://c9.io/) to play about with Clojure, according to this gist: https://gist.github.com/vothane/6678578

danp20:04:30

when I run lein repl it times out, but when I run the repl from java it works

noisesmith20:04:18

@danp what about if you run lein run -m clojure.main

danp20:04:22

Is there anything obvious that may be misconfigured

danp20:04:28

oh I'll check

noisesmith20:04:55

if clojure.main via run works, then the likely problem is permission to open a port - lein runs a client / server repl over a local tcp port

danp20:04:47

yes, that works - thanks @noisesmith

noisesmith20:04:04

@danp from that run your cp is set up and you can require your project

noisesmith20:04:38

so it's at least minimally usable - but you could also figure out what's happening with lein repl (eg. try using trampoline so it doesn't use a tcp port, or check configs for permissions)

danp20:04:01

ok, will give it a whirl - cheers

noonian20:04:36

You also might try increasing the timeout for lein in case its just slow to start: :repl-options {:timeout 120000}

noisesmith20:04:06

oh right - lein times out, I didn't think of that

danp20:04:19

Just trying with the timeout seting

danp20:04:44

Nope, nothing with the :timeout set

danp20:04:01

@noisesmith - lein trampoline works ๐Ÿ™‚

danp20:04:40

thanks very much - think I'll need to do do a bit more set up, but all seems dandy

noisesmith20:04:43

cool - then it's almost definitely a "permission to open a local port" thing

noisesmith20:04:01

which I could totally see being an issue in a container

danp20:04:11

yeah - to be honest, I think this is good enough. We're off on holidays next week and I want to be able to play around on the Chromebook ๐Ÿ™‚

noisesmith20:04:32

it just means no cider etc.

noisesmith20:04:49

wouldn't be a deal breaker for me, but I know some people really like their editor connected repl

danp20:04:49

i'm sure I can live with it for the week

danp20:04:36

Following on from my last issue, when I start lein with trampoline, I get this warning:

danp20:04:57

WARNING: cat already refers to: #'clojure.core/cat in namespace: net.cgrand.parsley.fold, being replaced by: #'net.cgrand.parsley.fold/cat

noonian20:04:12

That should be safe to ignore. Newer versions of Clojure added some new functions like cat and many libraries that define functions of the same name have not been updated to :exclude the Clojure functions which causes that warning.

danp20:04:38

ah, okay - nothing to worry about then ๐Ÿ™‚

donyorm21:04:47

So what's the story of lennigen vs. boot? What are the plus and minuses of each one?

mobileink21:04:26

donyorm: i won't address pros and cons, since it kinda broke my heart to move from leiningen to boot, but for the record they are completely

mobileink21:04:59

donyorm: different animals. sorry, fat fingered my phone.

mobileink21:04:49

leiningen is a build tool. boot is beyond category.

noisesmith21:04:21

lein has good docs and it's declarative and slow; boot is less documented and imperative and faster

noisesmith21:04:16

declarative vs. imperative: with lein you have configs that lead to build actions; with boot you write code that does your build actions

donyorm21:04:37

thank you for explaining that. I was about to ask the difference ๐Ÿ™‚

donyorm21:04:02

which one do you prefer using?

noisesmith21:04:37

I prefer leiningen - for a large part because its been around longer and its what I know, some day I might actually try boot

noisesmith21:04:57

if you are only going to know one build tool you'll have an easier time with lein, it is what most projects assume you use

tbaldridge21:04:59

You could also argue that lein is "easier" but boot is "simpler". There's a ton of magic in lein.

donyorm21:04:08

I'm having trouble deciding which one to go with, I'll have to read more on it

arohner21:04:03

As a quick generalization, use lein for bog-simple OSS libs that donโ€™t deviate from the norm. Use boot for big apps with complicated build/compile/deploy steps, or donโ€™t do things in the standard way.

mobileink21:04:26

donyorm: i won't address pros and cons, since it kinda broke my heart to move from leiningen to boot, but for the record they are completely

mobileink21:04:59

donyorm: different animals. sorry, fat fingered my phone.

mobileink21:04:49

leiningen is a build tool. boot is beyond category.

mobileink21:04:14

boot brings clojurity to task pipelines - groovy immutability and lots more. there's really no comparison except for trivial stuff.

mobileink21:04:19

i've worked with about a billion build tools, from make to cmake, scons, ant, gradle, etc. etc. and of course lein. they all suck, in their own charming ways. why? because they are build tools - they treat building sw as a special kind of thing. boot is the first and only tool i've ever seen that solves the build problem - by recognizing that building sw is just programming. no fancy dsl, just ordinary clojure code.

nikki22:04:41

just wondering -- what's the rationale behind require's syntax looking like (:require [x :as y] [z])? and not just taking some map / vec of the namespaces and aliases

nikki22:04:16

or not (:require [[x :as y] [z]]) (which is more lety)? i mean you have to pick something so everything will be quite arbitrary but just wondering

noisesmith22:04:53

let needs the extra [] to separate bindings from body

noisesmith22:04:59

require has no such distinction

nikki22:04:58

mmm good point

john22:04:14

If I was starting from scratch, I'd probably try to learn boot first, since it's more powerful in the long run. But yeah, I already know lein, and it works, so just haven't gotten around to boot yet.

mobileink22:04:15

ok i admit the learning curve is a little steep. but hey no pain no gain! is there no room for cliches?

mobileink23:04:00

the point being?

qqq23:04:32

lein/boot changes one's mindset about build tools

qqq23:04:42

with lein, my mindset was: is there a plugin taht does XYZ? no? ๐Ÿ˜ž

qqq23:04:53

with boot, the mindset is: I can program this in clojure/boot; it's just a matter of how