Fork me on GitHub
#clojure
<
2015-09-08
>
profil09:09:53

What do you guys use for plotting data?

joelkuiper10:09:43

R and Python 😉

joelkuiper10:09:27

we reuse a lot of plotting code from R and Python in more or less Polyglot approach. Interested to hear about pure Clojure stories though!

profil10:09:23

@jellea: That looks good! thanks

joost-diepenmaat11:09:35

@jellea hah. that looks amazing

joost-diepenmaat11:09:23

@profil: been messing about with incanter but I would not recommend it.

ghadi13:09:52

missing Clojure/Conj for the first time ever this year

ghadi13:09:11

super bummed

robert-stuttaford13:09:45

anyone had to share authenticated sessions across domains with a shared database? looking for the simplest way to do this. my research has folks doing OAuth2 all the way down to passing a session id on the querystring. i like the security of OAuth2 but it feels heavy considering we have a shared database

swizzard13:09:30

could you hash the username+password with a shared salt and store that as a cookie?

robert-stuttaford13:09:16

if i could use cookies i wouldn’t have the problem to begin with simple_smile

robert-stuttaford13:09:48

same browser, same database, same server, two domains. how to transfer authed session from one domain to the other?

cfleming13:09:33

@ghadi: BTW any PEG VM code to show off yet?

ghadi14:09:18

no PEG vm code yet. Hope to get some time this week

mchampine14:09:31

@robert-stuttaford: Instead of passing the authentication token in a cookie you can pass it in an auth header, cross-domain, via XHR. JWT is a popular token format with Clojure library support. http://blog.moove-it.com/token-based-authentication-json-web-tokenjwt/ . The standard login federation pattern involves double-redirect via OAuth, CAS, SAML or similar, but it sounds like your use case may let you bypass some of the complexity of those solutions. See also OpenID Connect which attempts to standardize JWT w/ OAuth (and much more). http://openid.net/connect/

robert-stuttaford14:09:06

thanks mchampine!

mchampine14:09:39

Good luck with it! Auth protocols seem to be in constant flux. Wish we'd settle on a few standard solutions.

robert-stuttaford14:09:59

me too. it’s a big field

robert-stuttaford14:09:35

JWT looks promising

robert-stuttaford14:09:40

@mchampine: looks like we could use https://github.com/liquidz/clj-jwt to provide a token safe to transport via GET from http://a.com to http://b.com, where http://b.com has all the creds to decode the token and use the values therein to find the relevant data in the database - does that sound right?

cfleming14:09:33

@robert-stuttaford: @mchampine: JWT looks pretty nice. One thing I don’t understand - how is it normally passed from the browser to the server? In a cookie? I’ve seen HTTP headers mentioned, but I don’t understand how those would be set.

niwinz15:09:02

It is usally passed in a Authorization header

niwinz15:09:22

Authorization: Token a7sf87g7rhrnn4n59923nun5...

cfleming15:09:14

@niwinz: So that means that it’s only good for requests from a JS client, right? i.e. if I were using a redirect or something I’d have to use a URL param?

niwinz15:09:18

It is usefull for anyone that want consume your api

niwinz15:09:40

passing it as query parameter it is also possible

niwinz15:09:11

but this has the same security implications as passing the http session token in the url parameter...

cfleming15:09:46

Right, what I mean is there’s no way to get the browser to automatically send back the Authorization header it was sent, or anything like that.

niwinz15:09:27

You can use the cookies...

niwinz15:09:52

cookies are sent automatically to all requests in the same domain...

niwinz15:09:06

or interdomain (for subdomains)

niwinz15:09:13

How you want or going to transfer the token, is not related to jwt. You can transfer it as you want.

niwinz15:09:51

In fact the original idea of jwt comes from stateless sessions...

niwinz15:09:02

that aready exists for years....

niwinz15:09:34

I have used something similar to jwt in the past when jwt does not exists. jwt only defines the proper way to secure sign/encode/encrypt the data (it standarizes it)... but it don't defines how you should use/transfer it.

cfleming15:09:05

@niwinz: Thanks, that was what I was really asking, even though I didn’t realise it simple_smile

timvisher15:09:13

@cfleming: if you put them in the resources directory then you can use resource lookup to get access to them and translate them to a file. assuming you’re using leiningen you could also just create a test-data directory or whatever at the project root and use relative paths.

cfleming15:09:00

@timvisher: Thanks, that’s what I ended up doing, following @nberger’s suggestion

timvisher15:09:25

oh man. i hate how slack doesn’t drop you to the bottom immediately…

timvisher15:09:27

anyone know all of the pre-reqs for a lein checkout project to be used?

timvisher15:09:48

i have a project with a checkouts directory with an absolute symlink to the project root of environ

timvisher15:09:14

i have tried running tests both without listing environ 1.0.0 as a direct dependency (it’s normally transitive) and without

mchampine15:09:42

@robert-stuttaford: yes, clj-jwt was what I was referring to for JWT in Clojure, and yes you'd send it as an Authorization header (or any custom header you choose) along with your HTTP request.

timvisher15:09:55

the version of environ in the project.clj of the environ source directory is 1.0.1-SNAPSHOT and the version that i’m directly depending on (or transitively) is 1.0.0

timvisher15:09:34

i have (for [x (range 100)] (println “you’re using the checkout”)) at the top level of environ.core, which is being used by the other project that depends directly on environ.

timvisher15:09:42

and i have further changes to environ

timvisher15:09:49

however, the printlns never print

timvisher15:09:02

and thus i do not feel confident that the checkout is being used. simple_smile

timvisher15:09:04

oh wait, for is lazy

timvisher18:09:00

is there any way to get pre-conditions to show the value that failed the assertion rather than just the form?

potetm18:09:28

@timvisher: Not that I know of, but if you find a way I would love to know about it.

lvh18:09:33

Hm. I have two projects; one where lein deploy deploys to clojars, and one where lein deploy errors out claiming not to be able to find dependencies. As a consequence, the default lein release doesn’t work. Neither project has :repositories set (I think). I have GPG creds in ~/.lein.

lvh18:09:13

Oh, wait, I was looking at the wrong project.clj. Derp.

amacdougall19:09:07

Anyone familiar with compojure.api.sweet and ring.mock.request? I'm trying to test a POST route:

; typed from memory, ignore mismatched parens etc, it _does_ build
(context* "/api" []
  (POST* "/login" []
    :return User ; this schema is defined earlier
    :body-params [username :- String, password :- String]
    (let [user (db/get-user-by-username username)]
      (if (user/valid-password? user password) ; user lookup and password validation are tested and working
        (ok user)
        (unauthorized)))))
I test it like this:
; app is the request handler in this case, of course
(app (request :post "/api/login" {:username "jrandomluser" :password "reindeerflotilla"}))
...but the response is 400, error "(not (map? nil))". I'm sure I'm doing something wrong, but I don't know if I'm defining the API route wrong or if I'm generating the request wrong. My next step will be to try hitting the API using curl or something, to rule out the request generation. I won't be able to work on this again until tomorrow night, so I'm still generating troubleshooting ideas... but if someone can immediately point out my bug, all the better.

nberger19:09:46

@amacdougall: from https://ring-clojure.github.io/ring-mock/ring.mock.request.html#var-request > (request method uri params) Create a minimal valid request map from a HTTP method keyword, a string containing a URI, and an optional map of parameters that will be added to the query string of the URI So I think you want to create the request without the params, then call body on it:

nberger19:09:48

(-> (request :post "/api/login")
    (body {:username "jrandomluser" :password "reindeerflotilla"})
    app)

nberger19:09:39

> (body request body-value) > Set the body of the request. The supplied body value can be a string or a map of parameters to be url-encoded.

amacdougall19:09:03

I did try that, but I'm afraid I got the same response. Also, the ring.mock.request tests imply that either syntax works—I think the body function is just to make it easier to alter an existing request. https://github.com/weavejester/ring-mock/blob/master/test/ring/mock/test/request.clj One of the tests:

(let [req (request :post "/?a=b" {:x "y"})]
      (is (= (slurp (:body req))
             "x=y"))
(And if it's relevant, when I did (slurp (:body req)) on the request I was making, I got username=jrandomluser&password=reindeerflotilla, which looks correct.)

timvisher19:09:11

how can i do with-redefs on a function defined with defn-?

amacdougall19:09:13

@timvisher: My gut tells me that since it's namespace-local, you can only do that if you're in the namespace. Is there a way of accessing defn-es from elsewhere? In some languages, like Ruby, the visibility control operators are more like suggestions...

timvisher19:09:33

@amacdougall: yes. but i don’t know if you can do that with with-redefs

amacdougall19:09:03

If you're in the namespace and still can't with-redefs it, then ¯\(ツ)

timvisher19:09:17

i’m doing it from a testing namespace

nberger19:09:13

Hummm, ok, then the docs are wrong... seems like the doc is valid for :get requests, but not for :post

amacdougall19:09:19

@nberger: Yes, I spent a good few hours yesterday being mightily confused about this. I still have some permutations I can use to narrow things down—including using a normal route instead of a compojure.api.sweet route.

venantius19:09:53

@timvisher: you should be able to using var referencing

venantius19:09:36

Eg #'var-ns/private-fn-name

timvisher19:09:48

@venantius: i’ll give that a try

venantius19:09:18

I've done that before to test private functions or otherwise to access them from another ns

nberger19:09:45

@amacdougall: ok. If I get to something I'll let you know. One thing: Not that it changes much (I checked this test and it looks the same), but I'd say you should probably check the ring-clojure fork, weavejester's fork is not maintained anymore

amacdougall19:09:04

Oh, that's really cool. So if I had something like this, it would work?

(ns private.core)
(defn- greet [] "hello")

(ns public.core)
(#'private.core/greet) ; returns "hello"

pbostrom19:09:14

@timvisher: I don't think you have to do anything special, with-redefs doesn't care that the var was originally private, it's going to redef it anyway

timvisher19:09:56

@pbostrom: it fails with messages like

Caused by: java.lang.IllegalStateException: var: e/read-env-file is not public

amacdougall19:09:15

@nberger: Good point! I just checked my project.clj and I do have [ring/ring-mock "0.2.0"], which is up to date. Thanks Luminus!

venantius19:09:26

@timvisher: this post covers the syntax and reasoning behind the var-reference syntax. pretty much exactly what you’re looking to do

pbostrom19:09:34

@timvisher: oh, you're right, you can redef it but to call it you'll need to #'

nberger19:09:40

@amacdougall: more on the debugging side ideas, you could try with enabling the swagger-ui and try from there... and then you can complement it with https://github.com/nberger/ring-logger to log what's going through exactly simple_smile

amacdougall19:09:21

Oh, that sounds useful! Someone very smart must have programmed that.

nberger19:09:28

BTW, I promise to stop posting the link to the project as soon as someone sends me some feedback 😛

amacdougall19:09:33

I'll let you know how it works! On the other hand, that means I'll have to figure out how logging works...

nberger20:09:40

If you don't want to figure it out, you can use ring-logger-timbre or ring-logger-onelog instead, those are separate projects depending on the first one

darwin21:09:07

trying to write a simple macro, which takes a param which should be a symbol bound to a string, I want to manipulate that string inside macro, how about to do that?

darwin21:09:31

here is my first trial: https://gist.github.com/darwin/b046cead5735de8840a8, which does not work of course, source gets evaluated as symbol name

darwin21:09:43

not as the string the symbol is bound to

gtrak21:09:12

why not do the calc inside the syntax-quotes?

darwin21:09:47

@gtrak this whole dance is to overcome defcard limitation

darwin21:09:07

it is a macro and it expects hard string as second param

gtrak21:09:11

crap, can't type that in here simple_smile

darwin21:09:36

I tried this one, didn’t work, let me double check it again

darwin21:09:15

btw. this is what defcard macro does, it tests for string? : https://github.com/bhauman/devcards/blob/master/src/devcards/core.clj#L59

darwin21:09:42

in general case I would need full eval of that symbol, but in my limited case, I just need to lookup the thing which the symbol is bound to and treat it as string, I was hoping to use &env, but my trials failed, need to read more about macros

gtrak21:09:14

ah, well you can call eval yourself on it.

gtrak21:09:20

but usually there's a better way

tom21:09:33

If I'm given 50 image links (let's say each is 4MB) and I want to download and write to files locally, is there any advantage to using core.async tools versus something like (pmap)?

gtrak21:09:14

50's a small number, I'd just make a future for each of them.

gtrak21:09:06

I wrote a toy recursive wikipedia downloader that used a few agents and a queue to do the same. You don't want to block the ioc threads with io, so in core.async you'd use the 'thread' macro if you're doing blocking io.

ghadi22:09:10

tom: pmap has no knobs, and won't properly use your cores in the way you'd expect (a straggling task will hold up progress)

ghadi22:09:34

there is clojure.core.async/pipeline-async with some knobs on concurrency

timothypratley22:09:22

core.async might give you some nicer error handling/retry expressions

timothypratley22:09:57

pmap uses the CPU bound threadpool, pcall uses the unbounded threadpool (typically these are called the send, and send-off pools)

timothypratley22:09:08

due to them being named so for agents

timothypratley22:09:26

the point being, pmap is for doing CPU intensive work, not IO

timothypratley22:09:40

if you use it for IO it will only have say 32 threads running at a time

timothypratley22:09:47

which is not what you want.

timothypratley22:09:59

for CPU tasks, limiting the amount of threads is good

timothypratley22:09:57

for example: (apply pcalls (for [file files] #(fetch file)) <-- will start downloading them all together

tom22:09:27

I'll look at pcalls more, but there's no advantage to involving core.async? Basically I'm trying to get these downloads done as quickly as possible and then ship them off for storage elsewhere.

timothypratley22:09:52

Right. You don't need core async to express that.

tom22:09:28

I have yet to find a good real-world use for core.async lol

nullptr22:09:08

tom: try doing some UI programming :)

max23:09:18

Hey everyone, I’m trying to expose a nrepl server using clojure.tools.nrepl.server. I’d like to be able to specify the namespace that it loads by default, and better yet, to insert some variable bindings into the repl session. Basically, I use dev/user.clj to define some useful repl tools for development, and I’d like to do something like that when I connect a repl to production. How can I manipulate clojure.tools.nrepl.servers session?