Fork me on GitHub
#clojure
<
2015-08-31
>
dominicm00:08:49

oh jeez, sorry, I was scrolled up.

dominicm00:08:01

I shall leave you alone now.

tel02:08:47

Is it possible to use Codox for a clojurescript/clojure mixed project?

tel02:08:54

all at once?

tel02:08:01

or do I need two documentation profiles?

casperc09:08:11

What would be the best way to calculate an etag for a ring resource? Is using the built in hash function in Clojure strong enough, or is there a faster/better way?

rickmoynihan09:08:20

casperc: Thats almost certainly not strong enough -- that returns the objects hashCode... which is used by java to store objects in hashmaps

rickmoynihan09:08:28

hashcodes aren't cryptographic - and can get a lot of collisions

rickmoynihan09:08:50

better to use a sha1/md5 etc...

rickmoynihan09:08:07

depending on your usecase

afhammad09:08:02

lein uberjar works locally but fails with "Uberjar aborting because jar failed: Subprocess failed" when run by dokku. Any ideas how to debug?

pesterhazy10:08:59

@afhammad: chances are the process it runs is java, right?

pesterhazy10:08:30

so maybe there's an issue about (the correct) java not being in the PATH

pesterhazy10:08:28

or, indeed, jar

afhammad10:08:36

@pesterhazy: So I tried this locally on a freshly installed dokku on vagrant, and it went smoothly, however its failing on digitalocean's default dokku setup

pesterhazy10:08:46

@afhammad: any way to get better debug logs (console output of the failed jar command)?

afhammad10:08:48

@pesterhazy: the output looks normal (as compared to running locally) and just before it would normally finish it ends with: Uberjar aborting because jar failed: Subprocess failed ! Failed to build. To <mailto:[email protected]|[email protected]>:app ! [remote rejected] master -> master (pre-receive hook declined) error: failed to push some refs to '<mailto:[email protected]|[email protected]>:app'

afhammad10:08:10

@pesterhazy: digitalocean support's first response are claiming it might be a RAM issue, and that i should upgrade..

pesterhazy11:08:26

@afhammad: that sounds like a pluasible explanation

pesterhazy11:08:59

you can try GNU time to measure the peak RAM usage of a process

casperc11:08:27

@rickmoynihan: Sorry, I missed your response. I know the hash function is not cryptographic, but it could be “good enough”, since you are basically just trying to avoid a collision with the last version of the resource served. It doesn’t need to be globally unique. md5 is a good option, but it is going to take some more CPU time to produce (I would suspect).

pesterhazy11:08:29

@casperc: the hashes won't be unique or identical across process restarts either

rickmoynihan11:08:19

@casperc: hashCodes are really only for putting objects in collections - you can't rely on the quality of the hashCode implementation

casperc11:08:36

@pesterhazy: Really? I thought they were created from the input only.

rickmoynihan11:08:40

@pesterhazy: they should definitely be the same across process restarts

rickmoynihan11:08:41

@casperc: regardless use a digest -- hashcodes can get a lot of collisions - depending on the object

pesterhazy11:08:16

@rickmoynihan: according to wikipedia, "[t]here's no requirement that hash values be consistent between different Java implementations, or even between different execution runs of the same program"

rickmoynihan11:08:39

you'll get really hard to track bugs otherwise... for example object foo and foo1 could easily generate the same hashcode... sure its unlikely -- but the 1 in a thousand time it does things will be very broken

pesterhazy11:08:02

@casperc: if you json'ify anyway, why not use the JSON representation as a base for the cryptographic hash?

casperc11:08:55

@pesterhazy: more for practical reasons. We use liberator, and that has a stream in the ring response at the point where we can hook into the request itself, so I would have to consume it.

rickmoynihan11:08:56

@pesterhazy: the contract is equal objects MUST have equal hashcodes... yes different JVM implementations may implement hashcode differently and I guess different executions might generate different hashcodes if they use e.g. object identity... so yeah when I said definitely I guess I meant 'usually' simple_smile

rickmoynihan11:08:23

regardless hashcodes should only be used for putting things in collections

dottedmag11:08:09

"Whenever it is invoked on the same object more than once during an execution of a Java application, the hashCode method must consistently return the same integer, provided no information used in equals comparisons on the object is modified. This integer need not remain consistent from one execution of an application to another execution of the same application."

dottedmag11:08:27

So it might differ after restart or in several processes.

dottedmag11:08:58

E.g. runtime might randomize hashCodes to avoid hash collision attacks.

dottedmag11:08:25

Oh, I see @pesterhazy already mentioned that.

plexus11:08:33

not sure what the JVM does, but some VMs base the hashcode of objects on their memory address, so they're definitely not consistent across runs

afhammad11:08:46

@pesterhazy: I guess they were right, upgrading to 1GB RAM fixed it..

pesterhazy11:08:37

@afhammad: uberjar shouldn't really consume that much RAM, but people probably just haven't run into this problem often because it's a short-lived process

cfleming12:08:48

@ghadi: What was the technical report you liked for the PEG VM approach?

cfleming12:08:56

Stupid 10k message limit...

cfleming12:08:07

(and stupid me for not bookmarking)

andrewboltachev13:08:11

Hi. If I need to have map with like 10 mln keys, would be in-memory Datomic DB more performant than just Clojure map? Just got java.lang.OutOfMemoryError: GC overhead limit exceeded, so, what rather should I use?

xificurC13:08:34

I'm building a project with a client/server architecture, i.e. a jetty server and a browser UI to work with it. I need the user through UI to select an arbitrary file and pass it over to the server. How is this done? If I understand correctly the server only sees stuff on its classpath. Does the file need to be POSTed to the server?

csmith13:08:22

xifi, Yes, you want a POST request. You’ll also need to use a multipart enctype form with an input tag using the type=“file” attribute. The backend just needs to handle that sort of request so that it can save the file somewhere

borkdude13:08:58

@xifi: if you're using ring-defaults you get good defaults

andrewboltachev13:08:05

Hi. How do I sent Clojure vec like ["foo", "bar"} via Clojure JNA to function that excepts char**?

ghadi13:08:42

@cfleming: that's the one.

cfleming13:08:13

@ghadi: Thanks, looks great from a quick skim

xificurC13:08:56

thanks @csmith and @borkdude . Yes I am using ring-defaults so I have the middleware running. Now just to tackle the form. Will the uploaded file be available on the classpath?

borkdude13:08:20

@xifi: it will be available in your params

borkdude13:08:35

@xifi: as a temporary file

csmith13:08:05

From there you save it to where you want, which could be on the classpath but you probably don’t want to upload code this way?

xificurC13:08:27

not code, data

borkdude13:08:38

@xifi: I'm copy pasting this from a project where I handle an uploaded file where input has name = "excel":

csmith13:08:42

(ok, was just checking)

borkdude13:08:03

excel is a hash-map where the file is available under the key :tempfile, just inspect it

xificurC13:08:20

thanks guys

pesterhazy13:08:47

@andrewboltachev: for the in-memory map, http://www.mapdb.org/ might be worth checking out

pesterhazy13:08:47

alternatively, sqlite is always good, or h2

andrewboltachev14:08:01

I wonder if Datomic would be good for 10mln

pesterhazy14:08:17

overkill if you're not using >1 clients

amacdougall14:08:02

Just noticed that metosin/compojure-api allows this:

(DELETE* "/user/:id" []
   :middlewares [audit-support (for-roles :admin)]
   (ok {:name "Pertti"}))
...so if your authorization logic is route-specific, it might make sense to reference it there.

juhoteperi14:08:45

@amacdougall: Here is also an example of using Buddy-auth with compojure-api: https://gist.github.com/Deraen/ef7f65d7ec26f048e2bb

amacdougall14:08:07

Oh, thanks! This is good stuff. ...I get it, compojure.api.meta/restructure-param lets you define your own params in terms of existing ones, so this ends up being a nicer syntax for using access rules as route-specific middlewares.

dominicm14:08:48

Is there any information on creating a clojure library, that's callable from java?

dominicm14:08:11

Without using the clojure java api, that is.

dominicm14:08:27

I'm imagining I'll have to gen-class or something, but I'm not sure.

dominicm15:08:05

And there's types to worry about, as it'll be called from Java.

dominicm15:08:07

@pesterhazy: That looks pretty perfect, yep. Thanks.

andrewboltachev15:08:39

Hi. How do I install library like this to use from Clojure? https://github.com/dren-dk/HunspellJNA I've installed it by "mvn install" but (import 'dk.dren.hunspell.Hunspell) raises java.lang.ClassNotFoundException

bostonaholic15:08:58

@andrewboltachev: if you're using leiningen, it will nee to be in your project.clj in order to be added to the classpath

andrewboltachev15:08:17

@bostonaholic: thanks, worked after adding to :dependencies

sdegutis15:08:03

What's your usual technique on having multiple versions of a service, (e.g. email), with an in-memory version for when running tests and a live version (e.g. AWS-SES) for production?

ghadi15:08:20

protocol providers

sdegutis15:08:04

@ghadi: What do you mean by that? Is that a common phrase?

sdegutis15:08:16

Can't find anything in Google for that phrase.

ghadi15:08:45

build some impls of a protocol like EmailProvider and make your namespace's functions take it as the first arugment

ghadi15:08:09

basically the old adage of programming to abstractions not concretions

sdegutis15:08:10

Oooh clever.

ghadi15:08:35

then it becomes an exercise in protocol design

sdegutis15:08:37

Actually I remember, that's the solution we used to have.

sdegutis15:08:06

The solution we have now is that there's a default in-memory implementation, just made up of normal vars, like (defn send-email [to from subject body] ...), and there's an alternative implementation in an adjacent namespace that actually uses SES, along with a function go-live! which just uses alter-var-root to replace the default implementation with this one.

sdegutis15:08:13

But it seems... weird.

ghadi15:08:38

you want:

ghadi15:08:54

(defn send-email [provider to from ... attrs])

sdegutis16:08:12

Ahh interesting idea.

ghadi18:08:18

not if the component legitimately has state to track

ghadi18:08:58

if you're doing something fishy with the atom, well then...

sdegutis19:08:13

What could cause a redefined var to not be noticed by live code?

sdegutis19:08:08

I ran (jetty/run-jetty #'my-handler {:port 8080}) and now inside CIDER I evaluated (defn my-handler [] {:status 200 :body "ok so far"}) with C-x C-e and refreshed my browser but the change is not visible. What am I missing or doing wrong?

tel19:08:24

Is there a way to overload get?

sdegutis19:08:34

tel: how so

tel19:08:49

I have a type which is not much different from a map

sdegutis19:08:56

@tel: one way is to make your type conform to IMap or whatever its called

tel19:08:00

but has some extra stuff attached which is largely ignorable

tel19:08:04

ah! there we go

tel19:08:30

oh, except that’s amusingly hard to google

tel19:08:05

is there a way to find that protocol?

tel19:08:22

wait, that’s just the runtime?

gtrak19:08:40

@tel: it's ilookup

gtrak19:08:49

get calls RT.get

tel19:08:49

mmk, thanks simple_smile

gtrak19:08:53

which casts to ilookup

tel19:08:05

gotcha gotcha

tel19:08:09

I felt like the “file” lookup was essentially get so I wanted to try not duplicating names

gtrak19:08:49

you can create your own namespace-local 'get' and hide the core 'get' if you want

tel19:08:58

yeah, thought about that one

tel19:08:02

it’s not a bad solution at all

gtrak19:08:06

implementing ilookup implies you're passing things around to code that already uses get generically

gtrak19:08:47

like, I implemented IOFactory on hadoop paths so slurp could work on hadoop input-streams, for example.

gtrak19:08:04

slurp had already been written for me

tel19:08:17

yeah, that’s very use directed

tel19:08:42

in my case, I just want to expose the fact that my filesystem is “essentially” a hashmap

tel19:08:58

that also can be descended into

tel19:08:12

it’s honestly two hash maps glued together

tel19:08:24

but you spend a lot of time operating locally

tel19:08:32

so, why not give preference to that?

gtrak19:08:42

Yea, i don't quite understand the api, but my first inclination would be to write the new functions anyway, then delegate Ilookup impls to those if convenient.

tel19:08:13

yeah, given the complexity I’ve already stepped into I agree with the approach you advise here

tel19:08:44

but I would certainly not cry if “local” work could be performed naturally by anything which expects a hashmap

bronsa19:08:03

clojure.lang.IPersistentMap

bronsa19:08:15

woah scrollback.

tel19:08:55

can I back into get via iterable or something like that?

gtrak19:08:47

ipersistentmap extends associative which extends ilookup

sdegutis19:08:29

@tel: Looks like you probably want to implement IPersistentMap

tel20:08:38

gotcha gotcha

tel20:08:41

thanks everyone simple_smile

tel20:08:10

@gtrak: also, whats an x-y problem? I’ve heard the term before but I’m not exactly sure what it means

tel20:08:02

ah ah, just googled instead simple_smile

audaxion20:08:05

so i have a list of maps that look something like this {:key “value” :type :type1 :some-other-random-key “another value"} and i want to end up with a list of lists like what partition-by returns, but i’m not sure how to construct my predicate so that the interior lists are grouped by :key and :type, but the overall order is maintained

audaxion20:08:41

anyone have ideas?

audaxion20:08:48

if that even makes sense

tcrayford20:08:47

,(partition-by (juxt :key :type) [{:key 1 :type 1} {:key 2 :type 2}])

audaxion20:08:25

that’s perfect!

audaxion20:08:30

thank you so much

erichmond20:08:14

Is your avatar Spike from Cowboy Bebop?

aengelberg22:08:19

For those of you who play League of Legends, I worked with Pepijn de Vos to make an item set builder in pure ClojureScript: http://pepijndevos.nl/2015/08/30/generating-league-of-legends-item-sets.html