Fork me on GitHub
#clojure
<
2015-12-14
>
davebryand04:12:50

hey all—can anyone tell me how I can (temporarily) edit the source of a lein dependency in my project? I basically want to throw a quick (prn) around a dynamic var that gets called in there

davebryand05:12:09

ok, looks like in ~/.m2 but I didn’t realize these are all jars ls -al ~/.m2/repository/eftest/eftest/0.1.0/. Is there even a way to edit source of something that was pulled in from maven?

shriphani05:12:33

hi, has anyone here used a fast disk backed queue? I don’t mind it if I need to use Java but I preferred something that works well with the clojure set data structure.

seancorfield05:12:31

@davebryand: If you want to run a modified version of a library on which you depend, the normal approach would be to clone the repo, modify it, lein install it locally, and depend upon that modified version (often libraries have a "SNAPSHOT" version on master).

davebryand05:12:11

ah, very good—thanks @seancorfield

Pablo Fernandez07:12:06

I’m working on a library and I need to test it on Heroku. Is there a way to do it without releasing an unfinished version to clojars?

jonas08:12:42

http://clojars.org seems to be down right now?

jonas08:12:08

clojars is back now simple_smile

borkdude09:12:34

Seems like the folks at Advent of Code reddit liked this snippet I made with loom: https://www.reddit.com/r/adventofcode/comments/3wj34t/graph_visualization_of_day_7s_wires/ Don't know if the author of the lib is here, but thanks for writing it simple_smile

robert-stuttaford10:12:11

anyone encountered this issue? Caused by: java.lang.NoClassDefFoundError: Could not initialize class clojure.tools.reader.reader_types__init

robert-stuttaford10:12:37

i’m sure i have the latest jar in my lein deps - 1.0.0-alpha2, and the latest tools.analyzer.jvm - 0.6.9

robert-stuttaford10:12:54

no clashes on either of them in lein deps :tree

robert-stuttaford10:12:20

driving me slightly mad 😐

robert-stuttaford10:12:35

only happens on our linux build box, fine on my mac

Pablo Fernandez11:12:30

How do I generate the pom and other files for release with lein?

borkdude11:12:01

@robert-stuttaford: have you tried lein clean?

borkdude11:12:52

@robert-stuttaford: and if that doesn't work, (re)move your .m2 dir and try again: lein clean + lein deps

borkdude12:12:25

(my god, can't believe I said: if that not works... such a Dutchism)

danielstockton12:12:32

i have a vector of vectors, is there a shorthand to map over the inner vectors like (map (fn [col] (map my-func col)) cols) ?

Pablo Fernandez12:12:05

flatten the vector and then map over it?

danielstockton12:12:06

i don't want it flattened in the end, i guess i'll just define a helper called map-inner or something

danielstockton12:12:10

thought it might already exist

Pablo Fernandez12:12:29

what do you mean by flatten in the end?

danielstockton12:12:53

nothing sorry, was wrong thinking

Pablo Fernandez12:12:55

(map func (flatten your-vec))

danielstockton12:12:07

that is a bit clearer, thanks

Pablo Fernandez12:12:38

Not sure if that works or whether it’s what you want. Does Clojure have flatten or was that Common Lisp?

danielstockton12:12:12

works marvelously, thanks

Pablo Fernandez12:12:32

Now, I think flatten is recursive, so if you have three levels of vectors, it’ll flatten them all.

danielstockton12:12:03

yep, it will, that isn't a problem

danielstockton12:12:57

hmm actually, it isnt quite the same as before

danielstockton12:12:20

i didnt check it properly

danielstockton12:12:42

do need it to only flatten one level

Pablo Fernandez12:12:05

Then you need to concatenate vectors.

Pablo Fernandez12:12:30

(map func (apply concat your-vector-of-vectors))

codefinger12:12:53

@pupeno you can use lein-heroku plugin to test your clojar withou releasing

Pablo Fernandez12:12:19

@codefinger: cool! simple_smile I’ll keep that in mind.

danielstockton12:12:05

@pupeno (inner-map inc [[1 2] [3 4]]) should return [[2 3] [4 5]]

Pablo Fernandez12:12:54

Oh… you want to preserve the structure of the vector. Sorry, I missed that.

Pablo Fernandez12:12:14

I’d just write a map call inside another.

Pablo Fernandez12:12:28

Not sure if there’s a better way.

jaen12:12:50

@danielstockton: it's probably an overkill for this use case, but you might want to know about - https://github.com/nathanmarz/specter

danielstockton12:12:41

thanks, looks interesting

Pablo Fernandez13:12:55

@codefinger: do you use it? uploading the build is taking forever.

codefinger13:12:20

@pupeno: no i made it :) it does have to upload your binaries, which can take a while depending on connection speed

eyelidlessness13:12:02

is anyone here using docker? is there a clear way to cache lein deps in a container?

mccraigmccraig13:12:25

@eyelidlessness: i tend to do the build separately from running the Dockerfile and just include the uberjar in the Dockerfile, but you could run lein deps as a step in your Dockerfile. the downside would be that when your deps actually change then you would have to throw your docker cache for the image away... which would perhaps mean having to run apt-get update etc again - not sure if or how you can remove just some steps from the docker cache

eyelidlessness14:12:12

interesting. i'll take a look at that. thanks!

borkdude14:12:27

How can I get Chrome to just show edn as text

borkdude14:12:35

instead of a file download

eyelidlessness14:12:40

pulling deps on each build is such a long wait, that i'd prefer to avoid it in prod (heck, i'd prefer to avoid it in dev)

mccraigmccraig14:12:45

hmm - lein has it's own mechanism for deciding when it's going to pull deps... can you tell lein not to check ?

eyelidlessness14:12:53

as far as i can tell it's using that mechanism, but from a fresh local repository every time

eyelidlessness14:12:19

i am trying out RUN lein deps right now, i expect it'll cache as expected

eyelidlessness14:12:31

@borkdude: i would be willing to bet there are a number of chrome extensions for that. but i couldn't guess how they handle scenarios where servers are not configured with the correct mime-type

lucien.knechtli14:12:10

couldn't you just mount the location that lein deps are stored as a data volume or something?

eyelidlessness14:12:10

that's interesting.

lucien.knechtli14:12:23

Haven't tried it, but it might work

eyelidlessness14:12:31

i'm new to docker. i'm getting the impression that volumes are the key to everything

lucien.knechtli14:12:58

yea they're pretty useful

lucien.knechtli14:12:45

I spent a few hours last week figuring them out. Turns out that selinux doesn't play very nicely with them

eyelidlessness14:12:07

one would hope that wouldn't be the case 😕

lucien.knechtli14:12:28

you get them to work together, but its a bit complicated

lucien.knechtli14:12:52

mostly the issue is that I wasn't getting any sort of notification from selinux that it was blocking docker

mpenet14:12:02

@borkdude: just set content-type header to text/plain I guess

mpenet14:12:20

or use curl if it's just for testing

jstew14:12:27

Everyone loves docker but me, it seems. For smaller deployments it's more trouble than it's worth to me. I can see where larger deployments would benefit but I don't deploy anything that big yet.

eyelidlessness14:12:11

i'm not 100% sold on docker yet, but i'm evaluating it

mpenet14:12:12

or use a server side switch, we have something like that where you can pass format=... to the request params (or via headers) and have it returned as json/edn/yaml etc

lucien.knechtli14:12:02

@jstew docker definitely has an upfront cost that is mitigated over time. If its a small deployment that's going to be in an environment, or even several environments, that you know everything about, it's probably not actually worth the upfront cost just as a deployment tool

lucien.knechtli14:12:45

I use it at work though, and it's definitely very useful when working on several projects at once with different requirements on the environment

mccraigmccraig14:12:22

@eyelidlessness: volumes are certainly useful - for my app containers i use them to mount the host syslog device /dev/log into the docker container so that i can use the host syslog to collect all container logging, and if you are using a container for a service with persistent storage then they are essential

jstew14:12:25

I haven't seen that up front cost mitigated over time though... The issue is doing things the "docker way". You can't just get a shell on your remote server to get a database dump, for example. You have to figure out how to do that through docker, then you end up either having to remember how to do that or write additional tooling to solve already solved problems.

jaen14:12:39

@eyelidlessness: why not build outside docker and then just put the resulting uberjar/war inside?

eyelidlessness14:12:49

@jaen: i'm, perhaps wrongheadedly, trying to ensure the entire build process has a predictable environment

jstew14:12:01

I have a couple of ruby apps deployed with docker. Nothing with the JVM though. It's been nice to have parity between local envs and staging/production but the JVM almost makes that a moot point.

jaen14:12:10

@eyelidlessness: this is not a bad idea, I suppose, but then I would suggest having a separate devlopment container (where you build the jar/war) and deployment container.

eyelidlessness14:12:27

that is what i have. you pretty much have to

jaen14:12:35

Deployment container doesn't need to have maven cache inside.

jaen14:12:53

And yeah, volumes can be useful to not have to re-download the cache over and over

eyelidlessness14:12:56

the only difference is i'm just taking the static build and deploying with nginx

jaen14:12:08

But docker's user management kinda makes it annoying

jaen14:12:34

(in that files will be created as root or given gid, and you can't easily map between users inside and outside of the container)

jaen14:12:08

@eyelidlessness: you can take a look at this if you want - https://gitlab.com/jaen/panda5

jaen14:12:31

It's basically the setup I describe with a dummy app

jaen14:12:36

No Clojurescript there with hot reloading, but it shouldn't be too hard to add either.

jaen14:12:13

My use case was "let Windows people run that without any hassle" which it unfortunately failed at (couldn't get docker running on Windows for some reason).

Pablo Fernandez14:12:32

@codefinger: oh! Thank you for making it then simple_smile

eyelidlessness14:12:49

@jaen: thanks i'll take a look

eyelidlessness14:12:23

re: windows, i can't say directly, but i do know that my OS X experience with docker has been world's better this time around than a year ago

jaen14:12:21

Well, it should be comparable, since both run VM, but somehow on Windows it was problematic to get it working; first docker couldn't see VT-x extensions (it seems you have to run it as an admin) and then something else.

jaen14:12:32

With linux there were no problems like that.

jaen14:12:27

If only they would fix the user mapping '

jaen14:12:43

It's annoying to get those files as root.

robert-stuttaford15:12:50

thanks @borkdude. i took another path: backing away from updating the deps in an old project simple_smile

borkdude15:12:58

@robert-stuttaford: simple_smile saw a talk about this on Clojure eXchange: 'Are you afraid of dependencies' (or similar title)

robert-stuttaford15:12:28

project has both server and cljs codebase. couldn’t get them to agree on a tools.reader.

Pablo Fernandez15:12:25

lein release changed the tag scheme

borkdude15:12:54

how do I write a 'like' clause in yesql?

borkdude15:12:40

same question for clojure.java.jdbc

jaen16:12:03

select from table where column like 'string';

jaen16:12:09

But I imagine that's not what you've meant?

pesterhazy16:12:38

In my GC logs I see [Full GC (Allocation Failure) 2783M->1890M(2784M), 17.0343879 secs] every few minutes or so. Am I right in assuming this means that the JVM stops the world for 15 seconds on each of those occasions?

pesterhazy16:12:51

My JVM settings include (in addition to -Xmx): -XX:+UseG1GC -XX:SurvivorRatio=1 -XX:NewRatio=2 -XX:MaxTenuringThreshold=15 -XX:-UseAdaptiveSizePolicy -XX:G1HeapRegionSize=32m -XX:+UseStringDeduplication

eraserhd16:12:32

I would like a clojure code search engine. E.g. I want to search for “(take (iterate &amp; ))” and see how frequently it occurs. Has this been done?

bronsa16:12:13

I remember something like that, yes, although I can't remember the name sadly

bronsa16:12:06

but I don't think it did that pattern-matching thing

bronsa16:12:20

no, crossclj just looks for var usages rather than for pattern usages

ghadi16:12:33

hey pesterhazy: it looks like you are using many jvm options, but you are not aware that that JVM can stop the world for 15 seconds. I recommend removing AdaptiveSzie and StringDedup

ghadi16:12:03

Go through this to tune the rest according to your particular usage http://www.oracle.com/technetwork/articles/java/g1gc-1984535.html

ghadi16:12:33

with GC simpler is usually better (G1GC will be the default soon)

eraserhd17:12:17

I wonder what would happen if one attempted to algorithmically optimize clojure.core’s API based on usage patterns. E.g. Inline fns and arities which are infrequently used and introduce fns and arities for common usage patterns.

sveri17:12:54

@eraserhd: Funny, I just read this: http://www.lispcast.com/100-most-used-clojure-expressions which is not exactly what you want but maybe the approach can be adapted.

eraserhd17:12:37

@sveri: Not a coincidence, I just read that, too, which is why I was thinking of it 😄

sveri17:12:48

haha, ok simple_smile

ghadi17:12:44

😃 sorry I couldn't find a nicer way to phrase it

ghadi17:12:54

GC pauses can be brutal

pesterhazy17:12:58

@ghadi: you correctly pointed out some cognitive dissonance

pesterhazy17:12:29

in my defense, I did not add these options originally but was asked to review them

ghadi17:12:33

yeah a lot of places I've worked have just had magic JVM_OPTS that have been passed down as tribal magic but no one ever understands

ghadi17:12:52

Look on the bright side: You are just about to be the expert!

pesterhazy17:12:06

before I read your comment I did the obvious and removed all the options except for -Xmx

pesterhazy17:12:31

it's a bit better but I'm still seeing 8s+ pauses, obviously totally unacceptable

pesterhazy17:12:40

there must be something in our code that generates tons and tons of garbage

pesterhazy17:12:11

@ghadi: I'll at least do my homeword and read Oracle's documentation on GCs next time

pesterhazy17:12:47

NOT the most fun part of working in the jvm however

sveri17:12:54

Hm, I am looking for memoization for an atom that accepts a TTL. I could do it manually like "delete after x ms", check on every deref it a value exists, if not, insert value, but that seems like a library could handle that better

ghadi18:12:01

pesterhazy: yeah. better tho than devops team just saying, "Hey I spent 10 minutes and found these magic opts online, can we try them?" Redhat+Oracle working on a new collector for giant heaps

ghadi18:12:24

from strangeloop 2014

alex.hall18:12:50

@sveri: Have you tried core.memoize?

Pablo Fernandez18:12:47

What’s the easiest way to listen for keyboard events in Clojure?

roberto19:12:51

what are u using? Swing or JavaFx?

sveri19:12:41

@alex.hall: Thanks, that works simple_smile

alex.hall19:12:34

yup, no need to reinvent that particular wheel simple_smile

jaen20:12:18

Should ns-tracker react to changes in cljc files?

robert-stuttaford20:12:45

it tracks files on disk, and might not yet have been updated to deal with .cljc

robert-stuttaford20:12:50

easy to find out on github

jaen20:12:44

Hmm... I can't find anything on github. tools.namespace though seems to support cljc

robert-stuttaford20:12:12

yeah, reloaded workflow (which uses tools.ns) is definitely better, overall

robert-stuttaford20:12:14

(have used both, definitely prefer reloaded simple_smile )

rantingbob20:12:49

Pardon my dumb, but what is the reloaded workflow?

robert-stuttaford20:12:26

a way to draw boxes around your app global state in such a way that you can easily tear everything down, reload all your code, restart all your stateful services, without restarting your jvm

robert-stuttaford20:12:07

i have an emacs function that invokes it, so: make some changes, hit C-c r, and i know everything’s properly recompiled (including dependent code) and cleanly restarted

robert-stuttaford20:12:21

takes a little upfront effort, but it’s so worth it

rcanepa20:12:57

Hi, I am trying to understand how can I control the http response (status & body) from a compojure-api API. My endpoint has a login function which can return a JWT in the form of a map {:token “…my token … “} and otherwise, I would like to respond with a 401 http code and some string message.

rcanepa20:12:35

I found in the docs that it can be done using some kind of case statement.

rcanepa20:12:03

However, I cannot completely understand how it works. Here is a link to the docs: https://github.com/metosin/compojure-api#response-models

rcanepa20:12:51

I can’t understand where the Total var comes from and most important, if this is the correct way of doing what I want.

juhoteperi20:12:29

@rcanepa: ok, forbidden and not-found are just functions from ring-http-response to create Ring response maps, case is just example implementation to return different responses

juhoteperi20:12:36

@rcanepa: Total var is a schema

rcanepa20:12:01

Ok, but on which result/variable is operating the case clause?

rcanepa20:12:13

On the return variable?

rcanepa21:12:02

If I understood well, this is the way I should do it, right?

juhoteperi21:12:46

@rcanepa: Yes. Also, if you only want to set schema for 200 response, you can use :return metadata handler

rcanepa21:12:59

@juhoteperi: Awesome… it worked perfectly! Thank you!!

jaen21:12:16

@robert-stuttaford: well, I have a pretty similar setup, just with this - https://github.com/danielsz/system/blob/master/src/system/boot.clj - and for some reason it can't recognize cljcs ; /

jaen21:12:18

And it also uses reloaded.repl.

roelof21:12:08

Is this one : http://rundis.github.io/blog/2015/buddy_auth_part2.html a good example how to make autherisation work