Fork me on GitHub
#clojure
<
2017-11-04
>
roti09:11:41

Hi, I'm experiencing a strange behaviour with cljs-http, I don't seem to be able to get the response values outside of the go block. Inside the go block everything's fine, but when I return a value outside of it, I get a #object[cljs.core.async.impl.channels.ManyToManyChannel]. Does anybody know what I'm doing wrong?

jumar09:11:12

@roti https://clojuredocs.org/clojure.core.async/go > Returns a channel which will receive the result of the body when completed

roti10:11:10

@jumar sigh, this means I can't make go return a normal value? If I need to put the value in an atom I'm loosing the decoupling (it's better than using callbacks, but still...)

joost-diepenmaat11:11:43

Go is not blocking so you cannot return a normal value. If you want that just leave out the go macro.

joost-diepenmaat11:11:54

Or possibly you just want to <!! The returning channel to block the outer thread.

Pontus12:11:31

Out of curiosity, does anyone know why if-some? isn't part of the language? I found myself writing (if (some? some-value)) pretty often and ended up writing a macro for it: if-some?. Just thinking that maybe I missed something, and that there's already a better way to accomplish the same outcome. Here's how I use it: https://github.com/pcolliander/api-test/commit/655d1347e085d5c688b8e2a38581ebfe0a2b53e9

ajs12:11:05

You could make the same case for many predicates, like if-every?, etc. doesn't make sense to bloat the lang API or mix abstractions like that.

ajs12:11:59

It's simple enough to combine them, your macro literally just adds one hyphen and removes two parens, a net one character change, right?

Pontus12:11:31

guess that's true, yeah just did it for readability

ajs12:11:36

Is the regular syntax hard to read? Adding macros often makes things harder to read because it's not always clear what the macros are doing. Always prefer the core language to writing new macros until the benefits of a macro are obvious.

roti13:11:33

@joost-diepenmaat yes, I realized that. my expectation was for go to be a full continuation, I guess that's not true.

roti13:11:02

i'm in the browser, so no <!! 🙂

jumar18:11:13

@pontus.colliander notice there's also if-some which is similar to if-let. you can also just use if if you can accept both nil and false

reefersleep18:11:59

Maybe it’s because the call to eval tries to resolve core/one and not foo/one?

abdullahibra18:11:47

@reefersleep so what do you think to make this clean in the code?

abdullahibra18:11:13

(def numbers '(foo/one foo/two foo/three))

phronmophobic18:11:37

it’s not that common to use eval

phronmophobic18:11:57

but when I do, i often explicitly set the ns when calling eval

phronmophobic18:11:01

something like

phronmophobic18:11:51

(binding [*ns* (the-ns 'foo)]
                         (eval numbers))

abdullahibra18:11:26

@smith.adriane thanks but can i you try this example at your machine

abdullahibra18:11:44

it's still not working

noisesmith18:11:52

that will result in an error if foo/one is not a function

noisesmith18:11:57

eval on a list calls it

noisesmith19:11:27

perhaps you want (map (comp deref resolve) numbers) but really hte important question is why you are even indirecting things this way

abdullahibra19:11:17

@smith.adriane thanks man done now using yours it's my fault

abdullahibra19:11:33

@noisesmith bad practice, i'm still working on this

phronmophobic19:11:02

you can use (def numbers '(list one two three))

noisesmith19:11:43

sure, that would get you the right answer, but using clojure's compiler to make your code work at runtime is like using a fork lift to carry your lunch box

noisesmith19:11:02

eval is a big heavy slow tool, meant for hard weird structural problems

phronmophobic19:11:08

sometimes it’s fun to drive around a fork lift 🙂

noisesmith19:11:09

not for getting some values from symbols

noisesmith19:11:39

if you want the performance penalty and code that is hard to reason about, go for it I guess

noisesmith19:11:01

but really at that point just use mutation, it's easier to wrap your head around and performs much better

noisesmith19:11:17

eval is a great way to get really hard to fix bugs

phronmophobic19:11:17

yea, i’m not suggesting to use it for production projects

phronmophobic19:11:43

but sometimes it’s fun to just tinker with

abdullahibra19:11:00

and sometimes it's a very big benefit using it

abdullahibra19:11:05

even in production

noisesmith19:11:19

no, what's the benefit?

phronmophobic19:11:37

I’ve used it for a project that is similar to the ipython notebooks

noisesmith19:11:32

the thing about this is that when eval results in a weird and hard to reproduce behavior in clojure, it's not because your notebook is poorly designed, its because the user fed it bad code

noisesmith19:11:33

but what I see people trying to do (like in this example here) is use eval to glue things together, and in that case you get a weird hard to weed out runtime error that is your fault, not the user's

phronmophobic19:11:37

agreed. using eval is almost never the right tool

phronmophobic19:11:44

there’s a lot of blog articles that talk about how cool lisps are because lisps have eval and macros. I think that’s why you sometimes get new folks to clojure trying those things out. on the one hand, using macros or eval is probably not the best solution to most problems, but on the other hand it’s a fun part of the language to play with if you’ve come from other languages that don’t make features like eval or macros as easy

noisesmith19:11:59

sure - you're making a compilation and code evaluation tool, that makes sense

abdullahibra19:11:00

@noisesmith well, what is the best practice in my situation above , how can do this ?

abdullahibra19:11:23

@noisesmith list with other vars in it and maybe i have deeper like this

phronmophobic19:11:27

(def one 1)
(def two 2)
(def three 3)
(def numbers [one two three])

noisesmith19:11:38

@abdullahibra if you need runtime lookup rather than compile time, you can explicitly look up the vars with resolve and dereference them

noisesmith19:11:51

it's cleaner to actually access the namespace as @smith.adriane shows there

abdullahibra19:11:08

that's helpful for me

mcall2521:11:13

I cannot reformat the json response (ring response) I am getting after a simple get request to my database. I would like fetch data from my database and formate the data into a map. below is the format I would like to get back from my json resonse in postman. Could someone please help format the response.? does any have an example project i Could look at? Below is the code flow.... [ {:first-name "mike", :last-name "tall" :team "football"}, {:first-name "nick", :last-name "tall" :team "york"}, {:first-name "ball", :last-name "tall" :team "sea"} ]

pwrflx21:11:01

hi! I'm trying to understand how to use "component". I've created a component to handle configuration variables. This is then given to the component that manages my MongoDb connection. However, how do I get the Mongo component in that namespace that is actually calling Mongo? I mean of course I could turn that namespace into a component as well, but it a seems too much for me, since this namespace holds no state overwise (it just takes the mongo connection as parameter and returns some data). Of course one way would be to store the system map in some central namespace as a var and in such cases reach out for that.. but that is explicitly discouraged in the component documentation.. so what to do instead?

noisesmith21:11:57

the component is a value that can be passed to functions

noisesmith21:11:16

you don't need to store it, your app can pass the components to things that use them

noisesmith21:11:38

in my production app, the mongo functions are passed a component that contains the mongo connection and db, you might even want to skip the component and just pass the connection, which should be a value in the component, to the functions that use it

qqq21:11:29

does 'float-array' return an immutable? if yes, what does NOT return an immutable if no, how do I set elements of a float-array ?

noisesmith21:11:55

there is no such thing as an immutable array, you can use aset

noisesmith21:11:12

see also amap, areduce, etc. etc.

qqq21:11:28

(def x (float-array (range 1000)))

(aset x 20 0)



qqq21:11:33

no matching method found aset

noisesmith21:11:12

+user=> (doc aset)
-------------------------
clojure.core/aset
([array idx val] [array idx idx2 & idxv])
  Sets the value at the index/indices. Works on Java arrays of
  reference types. Returns val.
nil

noisesmith21:11:29

0 isn't a float, that's the non matching method

noisesmith21:11:40

also why a float array instead of double array?

qqq21:11:57

hmm, 0.0 doesn't work either, I need (float 0.0)

qqq21:11:07

I'm interfacing with some gpu / ndarray stuff that uses floats

noisesmith21:11:08

right, 0.0 isn't a float, it's a double

pwrflx21:11:57

@noisesmith that's OK, but how does that code get the component that is calling the mongoDB functions? I mean my call chain is Jetty -> Pedestal -> Some business logic namespace that is pure functions -> MongoDb namespace. Currently I have a component for Jetty, Pedestal and Mongo

noisesmith21:11:40

your handler should be using the component, your app startup can deliver the component to the handler for example

noisesmith21:11:38

my component creates the handler function, and passes it to the function that starts the http server (along with some middleware that the component also creates)

pwrflx21:11:42

@noisesmith ok, so in my case, I should just make Mongo a dependency of the Pedestal component, and any time such a handler is invoked which requires mongo (eg in case of a "list of products" JSON REST service), pass in the Mongo connection ?

noisesmith21:11:37

yeah - and in fact you can pass in a subset of the component map - at some point you might also need a postgres connection, or some initialized resource

pwrflx21:11:43

@noisesmith hm. two problems with that. 1. In Pedestal, I'm not in a very direct relationship with the handler.. I mean there is a router in between, etc. So simply I don't see how I could pass the connection (or component) down the chain so it finally reaches that method where it is needed. 2. Actually MongoDb is not a dependency of my Pedestal component, but of my business logic. Eg just because Pedestal is restarted for some reason, there is no reason to also restart the MongoDb component

noisesmith21:11:10

I've never seen any reason to restart components piecemeal - it's fast enough that I just restart the whole thing. You can pass values that handlers need by attaching them to the request before it hits the router.

noisesmith21:11:53

the job of the component system is to ensure that all the states are coherent - starting and stopping as one big system is part of what makes that work

pwrflx21:11:34

@noisesmith ok, then I'll try to make it work this way 🙂 thanks!

pwrflx21:11:45

@noisesmith btw I'm also new to Mongo, is actually sharing a connection recommended? or should a connection be built up on every request?

pwrflx21:11:09

(I mean http request here)

pwrflx21:11:18

hm the https://github.com/danielsz/system/blob/master/src/system/components/mongo.clj creates a connection on component start, so I guess this is intended to be shared accross all users/requests (I was just wondering if there is sort of a similar thing like that's connection pooling to RDBMSs)

pwrflx21:11:09

ah ok, according to http://clojuremongodb.info/articles/connecting.html monger does connection pooling inside, so I can just use my single "connection" and this will get pooled by the library itself

qqq22:11:24

I need help optimizing this numeric clojure code. input: float array of size n output: float array of size (- n 1) where each output[i] = input[i] + input[i+1]

(def n 1000000)

(def x (float-array (range n)))

(def y (float-array (- n 1)))

(doseq [i (range (- n 1))]
  (aset y i (float  (+ (aget x i)
                       (aget x (+ i 1))))))


I'm on a GHz machine. loading + adding + storing 1M numbers should not take 10s of seconds how do I type hint / unbox / all types of crazy things ot make this fast ?

pwrflx22:11:14

@qqq no idea, but this benchmarking library is probably useful https://github.com/hugoduncan/criterium

phronmophobic22:11:33

(doseq [i (range (- n 1))]
  (aset-float y i  (+ (aget ^floats x i)
                      (aget ^floats x (+ i 1)))))

phronmophobic22:11:38

that’s a lot faster

phronmophobic22:11:49

there’s probably still room for improvement

phronmophobic22:11:13

as an aside,

(float-array
 (map (fn [a b]
        (+ a b))
      x (rest x)))

qqq22:11:15

@smith.adriane: that reduced runtime by about 99%

phronmophobic22:11:17

seems to work pretty well

qqq22:11:31

(the aset-float, ^floats solution)

phronmophobic22:11:53

the map version I think is a little more straightforward

phronmophobic22:11:15

albeit, slightly slower

phronmophobic22:11:36

but not nearly as as slow as the initial doseq version

phronmophobic22:11:17

(def y (float-array
         (map (fn [a b]
                (+ a b))
              x (rest x))))

qqq22:11:41

this is my fault for not clarifying this upfront, but I really have to modify an existing float-array (the 'posed problem' was a 'minimal case' of my 'actual problem'), so (float-array ... (map ... )) won't work

qqq22:11:58

the best so far is your solution of:

(doseq [i (range (- n 1))]
  (aset-float y i  (+ (aget ^floats x i)
                      (aget ^floats x (+ i 1)))))

qqq22:11:12

I wonder if there is a way to know of the + has boxing/unboxing involved

qqq22:11:25

so now we're at:

(set! *unchecked-math* :warn-on-boxed)

(def n 100000000)

(def x (float-array n))

(def y (float-array (- ^long n 1)))

(doseq [i (range (- ^long n 1))]
  (aset-float y i (+ (aget ^floats x i)
                     (aget ^floats x (+ ^long i 1)))))


and the remaining thing to do is to call disassemble and see what's goingon there

qqq22:11:29

unfortunately, disassemble is throwing an exception right now

mingp22:11:22

@qqq As an alternative, you could also consider writing this performance-critical section in Java and calling it via interop. (Passing on a suggestion from this channel not too long ago.)

zignd23:11:26

I'm getting an error in my Datomic setup that I'm not being able to get around. Here's what I have, a transactor running, a console connected to this transactor and a peer server connected to the same transactor. Whenever I try to connect to the peer server from the bin/repl found in the Datomic installation directory it works fine, here's an example output:

user=> (require '[clojure.core.async :refer (<!!)]
'[datomic.client :as client])
nil
user=> (<!! (client/connect {:db-name "clj-twitter"
:account-id client/PRO_ACCOUNT
:secret "mysecret"
:region "none"
:endpoint "localhost:8998"
:service "peer-service"
:access-key "myaccesskey"}))
#object[datomic.client.impl.types.Connection 0x2a235b8e "#datomic.client.impl.types.Connection[{:t 63, :next-t 1000, :account-id \"00000000-0000-0000-0000-000000000000\", :db-name \"clj-twitter\", :db-id \"datomic:dev://localhost:4334/clj-twitter\", :timeout 60000}]"]
But it doesn't work that well when I try the same from the REPL inside my project:
user=> (require '[clojure.core.async :refer (<!!)]
#_=> '[datomic.client :as client])
2017-11-04 21:21:22.610:INFO::nREPL-worker-0: Logging initialized @9976ms
nil
user=> (<!! (client/connect {:db-name "clj-twitter"
#_=> :account-id client/PRO_ACCOUNT
#_=> :secret "mysecret"
#_=> :region "none"
#_=> :endpoint "localhost:8998"
#_=> :service "peer-service"
#_=> :access-key "myaccesskey"}))
ClassNotFoundException org.eclipse.jetty.util.thread.NonBlockingThread java.net.URLClassLoader.findClass (URLClassLoader.java:381)
One thing I noticed is that the REPL from the Datomic installation directory is using Clojure 1.9.0-alpha15, while my project is using Clojure 1.8.0. Does anyone knows or have an idea of what I'm possibly doing wrong?