Fork me on GitHub
#clojure
<
2018-04-12
>
seancorfield00:04:40

@markw you can drop the apply in url-encode

markw00:04:54

oops - yeah you're totally right. Had that in there on an earlier version where I hadn't called str/join yet

benzap00:04:09

Is there a special way to package a library so as to leave out the clojure version within leiningen?

hiredman01:04:02

yes, the normal uberjar process does that

hiredman01:04:55

if you are getting a clojure version you don't expect packaged, you should check the output of lein tree :deps, likely some dependency is pulling in another version

hiredman01:04:48

a lein project doesn't come with clojure by default, either you have to depend on, or you get whatever random version one of your dependencies pulls in

benzap01:04:24

Oh, I mean is there a special way to package libraries so as to leave out clojure, but keep it around for development? From what you describe, you don't need to?

benzap01:04:00

ah, I won't worry about it for now 🙂

benzap01:04:01

Alright, i'm doing my social media rounds https://github.com/benzap/fif

seancorfield02:04:18

@benzap When libraries are published to Clojars, they only have the library code -- users run tools that bring in all the dependencies. Is that what you want? Or some middle ground between JARs and uberjars?

benzap02:04:24

@seancorfield Thanks, that clears things up

gklijs05:04:55

@seancorfield that would probably include the generated pom.xml right? But not the project.clj itself?

ajs08:04:42

will lein uberjar compile .java files before attempting to compile .clj files (assuming aot is on)? For some reason, I have src/java files that work fine when running a repl, but during lein uberjar it claims it can't find the java classes on the classpath.

ajs08:04:32

explicitly adding "javac" to the :uberjar :prep-tasks solved it

hmaurer13:04:31

Hello! Quick question: is there a standard way in clojure to attach test-cases to functions? So as to be able to list all the test cases related to a specific function

hmaurer13:04:39

I know I could do this myself through metadata

hmaurer13:04:46

but I am wondering if there is a standard approach

gnejs14:04:50

interesting.. it should be possible to introspect the test-code to see which functions it calls.. there should be a Leiningen plugin for that, no? 🙂

jjttjj14:04:02

If I need to parse a string representing a number in clojure, is it generally the best idea to parse it as a java Long instead of an Int even if I know it will be java int sized? since it pretty much will be promoted to Long if I do anything with it anyway?

Alex Miller (Clojure team)14:04:20

yes, parse it as a long

✔️ 4
danm14:04:28

Makes me sad when I'm using Java interop and the Java function requires an Int so I have to manually coerce my Clojure number to Int as part of the call

👍 4
eraserhd14:04:17

@hmaurer I just ran across a macro to do this that I was surprised to find. Was it in clojure.test?

eraserhd14:04:22

Yes, with-test, e.g. (with-test (defn foo [] 42) (is (= 42 (foo))) ...)

hmaurer15:04:20

oh wow, interesting

hmaurer15:04:54

thank you!

eraserhd14:04:23

I've always been fond of tests appearing with the code itself, and have never been sure why nobody likes this idea.

Alex Miller (Clojure team)14:04:33

@eraserhd I believe clojure.test already does this - you can attach test functions with :test meta in a defn and it will find and run them

Alex Miller (Clojure team)14:04:55

user=> (require '[clojure.test :refer :all])
nil
user=> (defn ^{:test (fn [] (is (= 0 1)))} foo [] (println "hi"))
#'user/foo
user=> (run-tests)

Testing user

FAIL in (foo) (NO_SOURCE_FILE:7)
expected: (= 0 1)
  actual: (not (= 0 1))

Ran 1 tests containing 1 assertions.
1 failures, 0 errors.
{:test 1, :pass 0, :fail 1, :error 0, :type :summary}

arrdem15:04:09

yeah you can implement and use all those structures in Clojure but you don't have a proof engine checking your work so their power to weight ratio isn't as good.

eraserhd15:04:51

My experience agrees with @arrdem's here. Without types, there is almost always a less verbose way to do these things. e.g. some kind of threading operator instead of a monad instance. In a typed language, the implementation is selected based on the type, and it isn't so verbose.

bronsa15:04:24

I mean, you don’t really have that in haskell either, the monad laws are satisfied by most monads by convention rather than formally verified

bronsa15:04:25

the reason why using monads/cat theory concepts in clojure doesn’t map too well is, IMO, not because of the lack of a type system, but because of the fact that in clojure we don’t create new types/categories for abstractions, we reuse what we already have, and what we already have already has monadic operators defined in the language

bronsa15:04:38

I’ve been writing ocaml for 6 months now and IME monadic structures don’t emerge because of the type system, they emerge because I’m creating new types all the time (which are 2 orthogonal things)

ddellacosta15:04:25

the types in Haskell absolutely do reduce boilerplate when it comes to stuff like monads, applicative, etc.--just because you can’t prove the monad laws via the type system doesn’t mean that the type system isn’t providing a ton of assistance in making it far simpler to implement and use those kinds of algebraic structures in Haskell vs. Clojure, where everything has to be manipulated on the value level, and where libraries like those (IMHO) are mostly useless as a tool to simplify how you structure computations

ddellacosta15:04:23

that doesn’t mean monadic structures don’t exist in Clojure, just that you can’t use the type system to express anything about them

ddellacosta15:04:47

I mean, @bronsa I think we are saying the same thing, in part, but I do think it’s important to acknowledge that the type system does play a part here

ddellacosta15:04:57

or, lack thereof

bronsa15:04:51

>just because you can’t prove the monad laws via the type system doesn’t mean that the type system isn’t providing a ton of assistance absolutely, I was stating the opposite tho, having a type system helps but doesn’t cause the emergence of the pattern, it’s having tons of types that helps the monadic pattern emerge

ddellacosta15:04:21

ah okay, apologies if I misunderstood

bronsa15:04:05

i.e. I’m arguing that the causality link for the emergence/usefulness of monadic pattern is a coding style that emphasises the creation of ad-hoc types, and having or not a type system is an orthogonal issue (it clearly helps a ton and promotes the usage of monadic patterns, but it doesn’t cause the emergence of those patterns)

😲 4
ddellacosta15:04:59

what do you mean by “…a coding style that emphasizes the creation of ad-hoc types…” ?

bronsa15:04:31

in clojure we don’t create an ADT for whatever

bronsa15:04:35

we just reuse a vector or a map

bronsa15:04:42

we already have tons of tools to manipulate those

bronsa15:04:09

if we did create tons of different types/data structures I’m sure we’d see the emergence of monadic protocols in clojure too

ddellacosta15:04:11

ah okay, so you were more talking about Haskell when you wrote that

bronsa15:04:32

s/haskell/any language focused on types/

bronsa15:04:05

I think it’s the same for ocaml, altho the lack of typeclasses makes it a bit less common than in haskell

ddellacosta15:04:00

right, although isn’t the lack of purity in OCaml and other ML-family languages a confounding factor there too?

ddellacosta15:04:12

in any case, I don’t disagree

bronsa15:04:12

no I don’t think so

bronsa15:04:57

it’s just that functors are significantly more heavyweight to use than typeclasses so it’s a bit harder to express a polymorphic bind for example

bronsa15:04:30

so what usually ends up happening is that in ocaml one just creates monomorphic ad-hoc monadic operators for whatever module they’re defining

ddellacosta15:04:15

back to Clojure, the way I’ve always thought about it is: in Clojure these monadic structures already exist, but they are subsumed in the “one-off”/ad-hoc constructs we get to deal with various operations that would be handled by a monadic value in Haskell. Stuff like state manipulation, IO, etc. I suppose one could say that for any imperative language though

ddellacosta15:04:00

but it also suggests to me that these kinds of libraries awkwardly encoding a bunch of algebraic structures on the (mostly) value level are of dubious utility

ddellacosta15:04:53

(clearly “ad-hoc” is the word of the day)

Bravi15:04:47

is 22MB a normal size for lein uberjar output’s standalone version? I’ve got 1 single clj file that is just under 50 LoC

sveri15:04:49

This depends on your dependencies. You can open the jar and look whats inside.

Bravi16:04:47

looks like clojurescript is taking quite a bit

Bravi16:04:33

since I don’t need it in production, I guess I can exclude it from uberjar right?

sveri16:04:02

If you are sure that is the case, you can do that. It depends on your build tool how to do that.

sveri16:04:37

lein deps :tree also shows you the complete dependency tree.

danm16:04:10

Oh interesting. Most of our uberjars are similarly big for small scripts but I'd not considered that it'd be pulling in ClojureScript

Bravi16:04:43

I stripped out clojurescript and reagent and now it’s just under 5MB

👍 8
eraserhd17:04:59

You know, I'd like to get a feel for what people feel are "big" or "small" clojure projects.

eraserhd17:04:28

Like, for me, I did a 8,500 line Clojure system, 35% test code, and thought that was kind of mediumish. I'm working on a 13,500 system, with Clojure and ClojureScript, and it seems absolutely huge unwieldy.

eraserhd17:04:39

And I think of 500 or 600 lines as a medium sized library.

Alex Miller (Clojure team)17:04:06

there are a number of Clojure code bases I’m aware of in the 50k+ range and a few in the 100k+ range

eraserhd17:04:34

Do people think of them as big or hard to deal with?

lmergen17:04:47

tbh 100k of clojure code easily compares to a 1MM+ java project, and they are also rare

hiredman17:04:01

I been working in 50k+ clojure code bases for several years now. my first clojure job had much less than that when I started, and and had blown past it several years before I had to find a second clojure job, that I am pretty sure was past that when I started (a little more than a year ago)

mccraigmccraig17:04:54

what are y'all using to measure loc ? i've got an ~80kloc codebase by a simple measure, and it doesn't seem unwieldy

👍 4
eraserhd17:04:04

@hiredman Ok, so when does a project typically feel "big".. Sluggish, hard to change.

val_waeselynck17:04:37

When stuff got complected?

👍 4
hiredman17:04:23

I am not sure you can directly compare like that

eraserhd17:04:40

I use find src test dev -name '*.clj*' -o -name '*.edn' |xargs wc -l.

eraserhd17:04:59

Well, if there's no pattern, and it's solely about design, that would be a thing to learn.

eraserhd17:04:03

I think that's probably true.

mccraigmccraig17:04:23

ha, quite similar to mine: find . -name \*.clj -o -name \*.cljs -o -name \*.cljc | grep /\*/src/ | xargs cat | grep "^.*\S.*$" | grep -v "^\s*;.*$" | wc

dpsutton17:04:26

one of our projects is 54kloc clojurescript

hiredman17:04:21

the first 50k+ code base left me wanting something with a clearer more defined module structure, so I use component as a basis for everything now

hiredman17:04:24

the first project had a kind of home grown module system, but it was built around global state (mostly def'ed atoms), which ultimately made testing and reusing code fraught

john17:04:52

Back when I was building hobby systems in clojure, I recall that things can quickly start to feel "too big" when I was using state management methods that made things too complex

mccraigmccraig17:04:25

we use multiple modules, a component system and a hexagonal architecture pattern... it seems to scale very well

eraserhd17:04:02

Weirdly, I suspect component makes things worse. We have a tangled set of dependencies and the system is awkward to boot for tests.

hiredman17:04:35

I'd say the first instinct of most is to organize code in some kind of taxonomy tree embodied in the file system. which can sort of kind of work, but you run into the common issue with taxonomies, of things not fitting cleanly one place or another

eraserhd17:04:39

hexagonal would be very nice, I think 😄

john17:04:22

Nowdays there's lots of workflows for keeping large systems sane

tbaldridge17:04:13

Tests are always awkward if they depend on IO, imo. Almost every project I've worked on has a pile of macros, configs that you have to cargo-cult to get a new test implemented. Component systems help keep it sane, but it's always awkward.

hiredman17:04:42

larger systems also require a different mental discipline to ignore the parts you aren't working on

🙈 4
hmaurer17:04:27

I am curious: does anyone here have a story to tell / an opinion on Clojure as a language for domain modelling, in comparison to, say, Haskell?

john17:04:52

I don't have much experience with Haskell, but can you get any more flexible for domain modeling, in a mapable and reducible way, than with clojure's maps?

john17:04:59

I guess if your model is a rule system, a type system would be more efficient

Alex Miller (Clojure team)17:04:32

I find in general, Clojure code is almost always amenable to being made smaller with the application of time and effort

8
👍 4
Alex Miller (Clojure team)17:04:51

and in some sense I appreciate having the choice about where and when I choose to work on that

Alex Miller (Clojure team)17:04:31

the key here being that improving abstractions in Clojure generally makes code smaller. Improving abstractions in Java generally makes code larger (by adding additional interfaces etc)

datran17:04:47

when using cognitect.transit/write, is there a way to produce a transit value? Or do you have to write to a ByteArrayOutputStream?

datran17:04:00

On the cljs side I see you can just do (writer :json), but on the clj side you need a (writer out :json)

noisesmith18:04:41

@datran there's no such thing as a "transit value"

noisesmith18:04:48

you want a string, right?

datran18:04:29

It looks like a vector in my repl, but I can believe it's actually a string

noisesmith18:04:52

I have no idea what kind of value you are talking about

noisesmith18:04:03

transit is a way to encode clojure data into bytes

noisesmith18:04:22

in cljs, you don't deal with bytes directly and instead you get strings

datran18:04:33

oh, okay, that makes sense

ddellacosta18:04:35

(which I think is JSON in practice, right?)

noisesmith18:04:45

the reason it gives you bytes is if you used a string, you'd still make some conversion into bytes to send the data to someone else (or store it)

noisesmith18:04:55

so it's actually more direct to just get bytes for the common case

noisesmith18:04:18

@ddellacosta that's configurable, edn and messagepack are also supported

noisesmith18:04:23

the default is edn iirc

datran18:04:48

Okay, so on the server I can just turn it into bytes and the client will interpret that as a string

noisesmith18:04:08

@datran you may just want the String constructor (String. some-byte-array)

noisesmith18:04:41

@datran well it depends on how good your ipc is - it might hide the transport from you and need to get a string

noisesmith18:04:56

or if you're lucky, it can accept a byte array or an InputStream

datran18:04:58

I'm pushing edn back and forth across a websocket using transit. So on the client I'm writing (t/write (t/writer :json) {:msg "hey"}) and on the server I'm translating that back into edn with (t/read (t/reader (ByteArrayInputStream. (.getBytes data)) :json)))

datran18:04:44

Now I just need to complete the round trip by pushing it back into bytes and sending that across.

datran18:04:00

You've helped clear up a fundamental misunderstanding I had, so thanks for that

noisesmith18:04:17

you might want to check what kind of data your websocket library can provide or accept - you mightbe able to make transit write directly to the socket for example

noisesmith18:04:56

or on the back end read, read directly from the socket

noisesmith18:04:05

all that said, the various byte-array / input-stream / output-stream / reader / writer / string conversions are actually pretty cheap - the apis are kind of annoying but they make sense once you take it all in

noisesmith18:04:51

but transit defaults to the lower level case, I assume for the situations where data transfer is in a hot path and you can't afford the extra abstraction

ajs18:04:55

I usually call toString on the bytes from clj before sending over socket to read in cljs. Maybe not necessary?

noisesmith18:04:42

probably not - depends on what your socket library knows how to use

noisesmith18:04:46

in fact you could probablymake transit write directly to the socket and skip creating the byte array - not that you neccessarily want to, but it would probably work

noisesmith18:04:19

@ajs if you think about how most socket stuff is working under the hood, the common pattern ends up with clojure value -> output stream -> bytes -> string -> bytes -> output stream -> client

noisesmith18:04:46

when really in theory (with the right socket api) you could just be doing clojure value -> output stream -> client

ajs18:04:13

I just follow examples I see since the lower level stuff isn't clear on specifics.

noisesmith18:04:24

(that's a simplification since an OS level api that java doesn't let you use directly treats the output stream as bytes on both ends of the network etc. of course)

noisesmith18:04:08

@ajs and that's fine - if the abstraction works and keeps things simple enough for you to use it effectively and it isn't a performance problem (it's unlikely to be) you can continue as is

ajs18:04:33

For example @dnolen has a client server transit example repo for Ajax so I just follow that process in the absence of anything more enlightening

noisesmith18:04:47

the reason I started rambling on this topic was because of the question "how do I get transit values from this byte array" which opens up the question of how transit is actually meant to be used etc.

noisesmith18:04:12

but if converting to string makes your code comprehensible, that's fine, go with it

ajs18:04:05

The lines between transit, bytes, text, json, and edn are a bit vague in most tutorials or examples.

noisesmith18:04:16

also there's a lot of imprecision in the conversation around that stuff - for example when it's suggested above that somone is getting edn out of transit - likely what was actually meant was plain clojure data of the same types supported by edn, since transit is an alternative to edn (which optionally uses edn, but that shouldn't matter beyond a config switch in your code)

ajs18:04:31

When you print transit it doesn't look like bytes, it's readable, just not as readable as json, which further confuses, is it text? Is it bytes? Questions without answers on most sites.

Alex Miller (Clojure team)18:04:58

transit is a json string (which is an encoding of bytes)

noisesmith18:04:05

@ajs the idea of "printing transit" is a bit sloppy though - you probably mean a string made out of the outputstream written to by transit

noisesmith18:04:34

or on the frontend, the string it outputs directly (I keep being jvm centric..)

ajs18:04:06

Like with protobuf I know I'm dealing with bytes. No gray area. The socket requires an array buffer. But transit uses a byte stream but isn't really bytes in that way.

noisesmith18:04:56

perhaps what you want is a tutorial on bytes, outputstreams, inputstreams, and strings on the jvm?

noisesmith18:04:53

I learned that stuff by googling and finding stack overflow answers about how to turn an output stream into an input stream or a byte array etc. etc. and converting the result into interop - I wonder if there's a good blog post or something on this

ajs18:04:50

That would be a worthy read. I nominate you.

noisesmith18:04:33

hahaha I'll seriously consider it

noisesmith18:04:14

it's interesting, in c all those things would just be arrays, in js all of those things would just be strings, but in these other languages with more nuanced abstractions we end up on all these variations and conversions between...

noisesmith18:04:19

time to start writing

4
sveri18:04:54

Stringtyping^^

schmidt7318:04:51

I'm a little confused about Korma and was wondering if anyone can help? Is there a way for Korma automatically create the DB for me or do I have to create the SQL scheme seperately?

sveri18:04:04

@henri.schmidt IIRC you have to do that yourself. There are a few migration libraries for clojure that will handle that for you. Regarding koma I would advise you not to use it as it is a bit to limiting. I stick to clojure.java.jdbc directly.

schmidt7318:04:49

Okay which libraries are those?

schmidt7318:04:44

And also is there any good reference for the difference between entities, entity-fields and the such? All I want is a simple database mapping users to positional entities which contain coordinates, speed, direction, and a timestamp.

myguidingstar19:04:46

Walkable is a new library that can help you with such "mapping" (sql term is JOIN) https://github.com/walkable-server/walkable

schmidt7318:04:06

I'm obviously not very well versed with SQL.

sveri18:04:26

Hm, I see. I dont use any of them myself as I find it easier to handle SQL directly instead of working with an abstraction over it. I would head over to https://www.clojure-toolbox.com/ and search for migrations

myguidingstar19:04:58

@henri.schmidt the Duct framework provides a very nice enviroment to explore sql yourself https://github.com/duct-framework/docs/blob/master/GUIDE.rst

schmidt7319:04:43

Thanks @myguidingstar that is really helpful.

ajs19:04:38

question about :aot :all when I run lein uberjar -- sometimes when I do the clean and uberjar build, it logs every namespace as it gets compiled. Other times, it will only show maybe 3 namespaces with the "Compiling..." log. Why would this be? Is it cached builds for those namespaces?