Fork me on GitHub
#clojure
<
2017-03-08
>
qqq00:03:31

instead of writing

(defn my-add [x y]
...)
is it possible to somehow attach extra info to it?
(defn my-add [^:int x ^:int y]
...)

hiredman00:03:26

have you seen core.typed?

qqq00:03:51

I'm trying to build an alternative to core.typed / specrtrum by hackig on a linter

hiredman00:03:26

have you looked at tools.analyzer?

xiongtx00:03:55

Is there a more elegant way to implement java.util.Map in use case such as:

(deftype GenericAvroMap [^Schema s ^java.util.Map m]
  GenericContainer
  (getSchema [_] s)

  java.util.Map
  (clear [_]
    (.clear m))
  (compute [_ k remapping-fn]
    (.compute m k remapping-fn))
  (computeIfAbsent [_ k mapping-fn]
    (.computeIfAbsent m k mapping-fn))
  (computeIfPresent [_ k remapping-fn]
    (.computeIfPresent m k remapping-fn))
  (containsKey [_ k]
    (.containsKey m k))
  (containsValue [_ v]
    (.containsValue m v))
  (entrySet [_]
    (.entrySet m))
  (equals [_ other]
    (.equals m other))
  (forEach [_ action]
    (.forEach m action))
  (get [_ k]
    (.get m k))
  (getOrDefault [_ k default]
    (.getOrDefault m k default))
  (hashCode [_]
    (.hashCode m))
  (isEmpty [_]
    (.isEmpty m))
  (keySet [_]
    (.keySet m))
  (merge [_ k v remapping-fn]
    (.merge m k v remapping-fn))
  (put [_ k v]
    (.put m k v))
  (putAll [_ m2]
    (.putAll m m2))
  (putIfAbsent [_ k v]
    (.putIfAbsent m k v))
  (remove [_ k]
    (.remove m k))
  (remove [_ k v]
    (.remove m k v))
  (replace [_ k v]
    (.replace m k v))
  (replace [_ k v new-value]
    (.replace m k v new-value))
  (replaceAll [_ f]
    (.replaceAll m f))
  (size [_]
    (.size m))
  (values [_]
    (.values m)))

xiongtx00:03:29

Without using a library such as potemkin: https://github.com/ztellman/potemkin

tbaldridge00:03:27

what's wrong with that code ^^

tbaldridge00:03:23

Proxy does something like that, but I'd be tempted to just use the code you have here

xiongtx00:03:26

OK, it’s just that Java’s interfaces are hideous then 😄

fellshard00:03:01

They really are. 😞

bcbradley01:03:39

how does a fully qualified class name work in clojure, if say, i'm depending on a java library from maven central like org.jogamp.joal/joal

bcbradley01:03:32

supposing I wanted a specific class, what would its fully qualified class name look like in clojure?

mtkp01:03:10

@bcbradley it would look just like in the documentation, for example, the first class in that list: com.jogamp.opengl.math.geom.AABBox

bcbradley01:03:21

thanks alot!

bradford02:03:57

Hi fam. What's the best server for doing server-side events @ 200k connections/server? (sparse traffic per connection). httpkit?

bradford02:03:54

And by "best" I mean "most straightforward implementation"

joshjones02:03:44

@bradford http-kit is very straightforward, and my testing has shown it to be quite good when compared to other clojure http servers. You may want to give this a read, as depending on your hardware, you may need to tweak some settings to increase the max file descriptors and ports http://www.http-kit.org/600k-concurrent-connection-http-kit.html

bradford02:03:57

@joshjones Thank you! Do you know if the example at https://groups.google.com/forum/#!topic/clojure/DbBzM3AG9wM is the 'current' way of implementing SSE, or is there a more up-to-date example?

tbaldridge02:03:03

I'd also look at Pedestal, may take a bit of testing, but there's no reason why it couldn't handle that many connections. And the Pedestal SSE implementation is pretty mature.

tbaldridge02:03:41

My biggest complaint with http-kit is that it's 100% custom code (last I checked). I prefer to use systems based on better established base libraries.

tbaldridge02:03:46

but that's just my opinion

tbaldridge02:03:23

The biggest issues you'll encounter with any solution however is memory usage (can't have a thread per request), and configuring the OS. So it seems more of a configuration problem than anything else.

tbaldridge02:03:49

Whatever option you go with, I don't think there's a out-of-the-box solution that will give you 200k connections, as that use case is pretty rare.

bradford02:03:29

I think most modern libraries like http-kit do something besides one-OS-thread-per-connection, I'm not quite sure the internals. I know some very high concurrency httpkit deployments, though.

yedi02:03:34

hi guys, am i doing something really stupid here?

yedi02:03:36

this (->> form #(conj % base-rect)) seems to be different than (conj form base-rect)

yedi02:03:45

the first returns a fn instead of the value that the second returns

yedi02:03:54

i thought i was using the threading macro correctly

tbaldridge02:03:24

@yedi, call (macroexpand '(->> form #(conj % base-rect))) and compare the results

seancorfield02:03:15

and then (macroexpand '(-> form (conj base-rect))) and see what you get

tolitius02:03:25

@bradford I would give immutant and aleph a try as well if you have time to experiment

yedi02:03:06

@tbaldridge @seancorfield so i did that, and im still not really sure i understand why it works that way

yedi02:03:14

tho i can't really pinpoint what im confused about

bradford02:03:02

wait, you can use the % with threading macros? whoaa

yedi02:03:11

u can't apparently?

yedi02:03:57

like in this case (macroexpand '(-> form (conj base-rect)), why does conj properly get placed in front of the argument we want (when it generates the fn call)

yedi02:03:23

but if you were to use an anon function it doesn't just place the entire anon fn in front of that argument and call it

joshjones02:03:11

Short story -- when you use anon funcs in a thread macro, surround it with ()

joshjones02:03:24

(->> [42] (#(conj % [99])))

seancorfield02:03:46

@yedi #(conj % base-rect) is short for (fn [x] (conj x base-rect)) so when you ->> into that, you get (fn [x] (conj x base-rect) threaded-var-goes-here)

seancorfield02:03:44

As @joshjones says, you need to wrap the (anon) function in paren to get the result you want

seancorfield02:03:38

(->> form (#(conj % base-rect))) => (->> form ( (fn [x] (conj x base-rect)) )) => ( (fn [x] (conj x base-rect)) form )

seancorfield02:03:16

(but in this case you’re better off just doing (-> form (conj base-rect)) )

yedi02:03:40

ah, i wonder why the macro doesn't take the surrounding parens in the fn definition into account when generating the expanded form

seancorfield03:03:36

(->> form (fn [x] (conj x base-rect))) => (fn [x] (conj x base-rect) form) — that’s exactly how threading is defined.

seancorfield03:03:14

Consider that (macroexpand '(->> 1 (2 3 4))) => (2 3 4 1) — pure symbolic substitution, not necessarily a valid expression you can evaluate.

yedi03:03:57

hm what about the scenario when you're just (->> x inc)

yedi03:03:03

it automatically adds the parens around inc

yedi03:03:05

to get a fn call

seancorfield03:03:35

Yes, the special case is if the form has no ( ) they are added.

yedi03:03:42

ahhhh i see

yedi03:03:50

that can be a lil confusing but i get it now

seancorfield03:03:06

so (->> x inc) is first transformed to (->> x (inc)) and then to (inc x)

seancorfield03:03:21

boot.user=> (macroexpand '(->> thing (some [arg] (an arg expr))))
(some [arg] (an arg expr) thing)

seancorfield03:03:55

Macros “don’t care” what the symbols are — they don’t get any meaning until after macro expansion (in general).

bcbradley03:03:06

What is the first argument to ns?

bcbradley03:03:09

is it a symbol?

seancorfield03:03:20

ns is a macro so it interprets raw s-expressions...

bcbradley03:03:30

i'm trying to call ns programmatically

bcbradley03:03:42

i'm processing a collection that describes an api

bcbradley03:03:47

and it has multiple packages

bcbradley03:03:00

period isn't a valid token for a symbol

seancorfield03:03:06

You'll want to call create-ns and stuff like that.

bcbradley03:03:54

suppose my package is clojogl.graph.curve

bcbradley03:03:12

i would want to say (ns (create-ns 'clojogl.graph.curve)) right?

bcbradley03:03:21

but idk it doesn't seem right

seancorfield03:03:45

You don't need to call ns at all. It just expands to calls to other functions -- you should call those directly.

bcbradley03:03:04

what about the symbol argument to create-ns

bcbradley03:03:12

it has periods in it, but that isn't a valid token for a symbol is it?

seancorfield03:03:08

(in-ns 'foo.bar.quux) is valid and creates the namespace and sets *ns* to that -- which is basically what ns does under the hood...

bcbradley03:03:53

then the documentation for symbol should be updated to say that valid tokens are alphanumeric characters and *, +, !, -, _, ? and .

seancorfield03:03:08

Do (macroexpand '(ns clojogl.graph.curve)) to see what it does behind the scenes

seancorfield03:03:14

It starts by calling in-ns

seancorfield03:03:54

What you can create as a symbol is different to what you can programmatically produce as a symbol. Same for keywords.

bcbradley03:03:18

thats a bit confusing

seancorfield03:03:37

For example (keyword "Hi Brian!") -- but you cannot type :Hi Brian!

seancorfield03:03:30

and (symbol "! @ # $ % ^ & * ( )") will produce a Symbol -- but you can't type it in literally.

bcbradley03:03:40

but i still don't see a period there

bcbradley03:03:47

thats what stumped me

seancorfield03:03:06

(symbol ".") is valid 🙂

bcbradley03:03:58

i'm hoping to make a wrapper around jogl so we can do some low level graphics stuff with clojure

bcbradley03:03:13

normally i'd just go through the java interop

bcbradley03:03:28

but i wanted docstrings and shebangs where they make sense

bcbradley03:03:05

also kebab case because i'm picky

seancorfield03:03:52

FWIW (def a.b 42) does work (and defines a symbol with a dot in its name)

bcbradley03:03:24

i guess i should ask the repl before asking it here

bcbradley03:03:34

or checking the docs

seancorfield03:03:01

Well, just because it's accepted doesn't mean it's "defined" per the docs.

seancorfield03:03:12

You can't do (let [x.y 13] ...) for example.

bcbradley03:03:11

if i may tap your experience

bcbradley03:03:24

can i use java reflection at the top level?

seancorfield03:03:25

So now I went and read the docs:

'.' has special meaning - it can be used one or more times in the middle of a symbol to designate a fully-qualified class name, e.g. java.util.BitSet, or in namespace names. Symbols beginning or ending with '.' are reserved by Clojure. Symbols containing / or . are said to be 'qualified'.

bcbradley03:03:36

i want to ask a java class what its methods are

seancorfield03:03:39

So it explicitly says . is allowed and has a particular meaning.

seancorfield03:03:24

(I only tend to consult the docs when something doesn't work 🙂 )

bcbradley03:03:27

i guess i could have chased the link to the reader docs though

bcbradley03:03:00

huh i didn't know that

seancorfield03:03:32

Both http://clojuredocs.org and http://clojure-doc.org are community-maintained. http://clojure.org is the "official word".

bcbradley03:03:58

ok thats nice to know

seancorfield03:03:34

Re: Java methods... I'd probably use Java's reflection stuff but I'd have to go read the docs... I try not to keep too much of the Java API Docs in my head 😐

bcbradley03:03:59

basically i want to look inside a package and ask it what classes there are

seancorfield03:03:02

Or maybe the Clojure reflection package if you wanted data structures... again, I'd have to go read those docs...

bcbradley03:03:05

then inside each class ask it what methods there are

bcbradley03:03:24

looks pretty straightforward

bcbradley03:03:41

what i'm worried about is whether everything is loaded in time

bcbradley03:03:54

i don't know if its valid to do this kind of reflection at this point

bcbradley03:03:53

my "structures" look something like this:

{"graph.curve"
  {"OutlineShape"
    {"addEmptyOutline"
      {:doc "Add a new empty Outline to the end of this shape's outline list. If get-last-outline is empty already, no new one will be added. After a call to this functiona ll new vertices added will belong to the new outline. "
       :args ["outline-shape"]
       :pure false}}}}

tech_hutch03:03:55

I don't know about Java, but you can get all Clojure namespaces as a sequence with all-ns.

seancorfield03:03:13

I don't know that you can start from a package name and find all classes that belong to it...? I think you're right that you need to load a class to be able to reflect on it.

seancorfield03:03:45

@tech_hutch That will only return the loaded namespaces I believe.

bcbradley03:03:18

i'm depending on org.jogamp.jog/jogl "2.3.2" of course

bcbradley03:03:52

i'm not sure whether or not i can actually ask a package what classes it has

bcbradley03:03:57

i know i can ask a class what methods it has

bcbradley04:03:06

i intend to do that, then do some text processing to pull out the camelcase name

bcbradley04:03:12

convert it to kebab, and make symbols out of it

bcbradley04:03:25

then programatically use defn inside the proper ns

bcbradley04:03:36

i've got :pure false there

bcbradley04:03:42

that means i'll add ! to the end of the method name

bcbradley04:03:52

and i don't think i can pull in the docstring, so i have to put that there too

seancorfield04:03:56

I guess you could read the JAR itself to get a list of all the .class files?

tech_hutch04:03:57

@seancorfield ohh. Is there any way to get the subnamespaces of a namespace in Clojure?

bcbradley04:03:26

i guess thats possible

bcbradley04:03:39

its likely i need to manually make most of the structure anyway though

bcbradley04:03:47

because it has to include some semantic information that isn't documented

bcbradley04:03:52

whether something is "pure" or not

bcbradley04:03:43

i just don't know if i can actually do reflection at this level because it don't know what is loaded when clojure begins evaluating

seancorfield04:03:50

@tech_hutch If you start a REPL in a project and immediately run (all-ns) it will only show the already-loaded namespaces -- it won't show any of the project's namespaces until they have been required.

seancorfield04:03:25

For example, I just created a new project -- boot -d boot/new new -t app -n hutch; cd hutch; boot repl -- and (all-ns) doesn't show the hutch.core ns even tho' hutch/core.clj is on the classpath and can be loaded.

seancorfield04:03:06

So, no, short of walking the class path and the filesystem, you can't find "all" namespaces (to include the not-yet-loaded ones).

bcbradley04:03:59

so you are saying i'd be better off generating the clojure code and making a library out of that, rather than asking clojure to generate the library at load time?

seancorfield04:03:51

@bcbradley I think that generating Clojure source from the Java classes in the JAR file is probably going to be an easier problem to solve -- and would let you tweak the generated code as well.

tech_hutch04:03:38

The docstring for all-ns just says "Returns a sequence of all namespaces." It could probably be more descriptive, then.

seancorfield04:03:42

Trying to generate Clojure stubs on the fly from an arbitrary Java library... sounds like an interesting problem to solve but I'm not sure how tractable (or performant) it would be...

seancorfield04:03:19

@tech_hutch I agree. And the source doesn't give you any more clues either.

bcbradley04:03:29

i guess i could treat it like an application rather than a library,

bcbradley04:03:36

since apparently i'm trying to make an application to generate a library

bcbradley04:03:57

in that case, i could invoke in-ns and all that jazz inside the main

bcbradley04:03:17

i think it would be guaranteed to be loaded by then right?

bcbradley04:03:51

i'm sure there is a clojure function somewhere that gives me the string representation of the functions in a namespace

bcbradley04:03:58

then i can take that and spit it to the filesystem

seancorfield04:03:56

If you load a namespace, you can get a list of symbols in it and call clojure.repl/source on those I guess...?

bcbradley04:03:20

god this feels dirty

tech_hutch04:03:24

There's ns-map.

seancorfield04:03:48

As for loading the jogl library classes, you'd need to do something to explicitly reference them, in order to get them loaded (as I understand things).

seancorfield04:03:08

If you don't reference classes, they don't get loaded.

bcbradley04:03:29

reference or instantiate?

tech_hutch04:03:12

Do you know of any online repls that have the latest version of Clojure? http://tryclj.com runs 1.4.

seancorfield04:03:29

Class.forName( someClassName ) will give you a Class object for the named class, if it can be loaded.

bcbradley04:03:47

thats very convenient thanks

bcbradley04:03:09

when you say "can be loaded"

seancorfield04:03:11

(Class/forName some-class-name) in Clojure 🙂

bcbradley04:03:29

do you mean "can be loaded right now and I won't make you wait"

bcbradley04:03:40

or "can be loaded if its on the classpath, and if it isn't loaded yet i'll load it and make you wait"

bcbradley04:03:01

if its the latter that simplfies everything

bcbradley04:03:05

i can just use that

seancorfield04:03:57

If it isn't on the classpath, you'll get an exception. Otherwise you'll get an instance of Class describing the class -- after loading it if it wasn't already loaded.

bcbradley04:03:33

so have you ever done this or are you just an expert?

bcbradley04:03:40

because i don't know any of this stuff

seancorfield04:03:53

I've been working on the JVM since 1997...

bcbradley04:03:12

i know who's shirt to tug on then

seancorfield04:03:14

I'm just old 🙂

bcbradley04:03:47

i've been wanting to make a game for clojure for a while

bcbradley04:03:55

but play-clj has some quirks

bcbradley04:03:16

and there isn't really a good low level interface to opengl except penumbra, and that isn't actively maintained

bcbradley04:03:20

and it is a little opinionated

seancorfield04:03:28

I started doing commercial software development back in the early 80's... C and assembler, COBOL, moved to C++ in '92, picked up Java in '97...

bcbradley04:03:49

what do you think of c++ now?

bcbradley04:03:58

i mean after they added gobs of whatever to it

seancorfield04:03:09

I was on the X3J16 Standards Committee for eight years ('92-99) and secretary of it for the last three... I guess I'm proud of what we achieved... and it's been interesting to watch it grow since I left the committee... I still chat with several of the active members.

bcbradley04:03:41

Really? Well I don't think c++ is that bad from a technical point of view

bcbradley04:03:58

in fact its probably the most efficient language from that point of view that I know of

bcbradley04:03:09

but the problem i had with it is that it elevated that concern above all others

bcbradley04:03:20

i guess there has to be a language that does though

tech_hutch04:03:28

I learned Java in college, but haven't had much use for it since then.

seancorfield04:03:42

The design goal was that you only pay for the features you use -- you should never pay a cost for features you do not use.

seancorfield04:03:53

That drove a lot of what we did on J16.

bcbradley04:03:53

but you do, its just not a performance cost

bcbradley04:03:55

its a developer cost

bcbradley04:03:59

still a cost

seancorfield04:03:03

Yes, it has gnarly syntax.

bcbradley04:03:40

i guess the other thing i don't really like about it is that it tries to be everyone's friend

bcbradley04:03:51

i mean it mixes ever paradigm out there

bcbradley04:03:01

sometimes too much variety is a bad thing

bcbradley04:03:10

langauges in my opinion should have a coherent focus

bcbradley04:03:23

i think that is very much antithetical to the design of c++ though

seancorfield04:03:36

It was designed as a multi-paradigm language from the get-go.

seancorfield04:03:48

Have you read The Design and Evolution of C++?

seancorfield04:03:36

It's a good read -- fairly accessible and not very long -- explains a lot about why C++ is the way it is.

tech_hutch04:03:50

Reminds me of JavaScript. Everything and the kitchen sink.

bcbradley04:03:53

i may look at it some time when i have some time to kill

bcbradley04:03:03

but i think i probably already know why it is the way it is

bcbradley04:03:16

it is committed to backwards compatability

bcbradley04:03:30

and at the same time, focused on supporting any new paradigm that arises

bcbradley04:03:58

without giving that paradigm ample time to prove or disprove itself in the wild before deciding whether to include it

bcbradley04:03:08

so you end up with a language that looks like the tree of life

bcbradley04:03:22

a small amount of live, active branches

bcbradley04:03:40

it makes the language hard to understand for newcomers because documentation could be obsolete in 5 years

bcbradley04:03:50

not in terms of whether something "will work" that way

bcbradley04:03:56

but whether it is considered idiomatic to do it that way

bcbradley04:03:29

when you have these two competing interests in a language it causes the idiomatic time frame to shrink

bcbradley04:03:46

something in c that was written 20 years ago that was idiomatic then is idiomatic now

bcbradley04:03:50

not so for c++

seancorfield04:03:04

Well, ISO and ANSI require that standards are reviewed and that updates are at least considered every five years as I recall...

bcbradley04:03:32

those standards reviews sometimes add a new thing to plain c

bcbradley04:03:36

like threads for instance

bcbradley04:03:44

but they don't change what would be considered idiomatic or not

bcbradley04:03:53

they don't add a new kind of pointer for instance

bcbradley04:03:14

i don't think we will ever see them add lambdas or such

bcbradley04:03:16

thats all i'm saying

bcbradley04:03:22

c++ wants to have it all

bcbradley04:03:26

and the developer pays for it

bcbradley04:03:36

(or not, in the case of people simply not bothering to learn it anymore)

bcbradley04:03:29

i'm being kind of unfair though

bcbradley04:03:46

what actually happens is different c++ programmers end up with a different understanding of what is idiomatic

bcbradley04:03:57

and there is little community consensus on it because of the momentum of the language

bcbradley04:03:08

so you end up with 5 experts in c++ working on something

bcbradley04:03:21

and they have trouble reading eachother's code moreso than they would in another language

bcbradley04:03:34

some of that trouble is naturally intrinsic to their different styles of programming

bcbradley04:03:57

but a large chunk of it is exacerbated by c++ culture, which is derived from the shrinking idiomatic window i described.

seancorfield04:03:27

You could levy the same complaint against Scala 🙂

bcbradley04:03:38

and even clojure some extent

bcbradley04:03:44

but i forgive growing pains

bcbradley04:03:47

c++ has no excuse

seancorfield04:03:27

I enjoyed writing C++ for 10-15 years. I adopted Java because, back then, it was a much cleaner, simpler, smaller language (and library).

bcbradley04:03:10

Well i'm sure it was enjoyable, or you wouldn't have done it for 10-15 years (i hope)

bcbradley04:03:29

I just have some opinions about c++, it nothing personal

bcbradley04:03:09

to me a language is the whole package

bcbradley04:03:11

community and all

seancorfield04:03:27

Java's library quickly grew out of control and, since Java 5, the language itself has sprouted all sorts of ugly stuff. I parted ways with Java round about Java 5. The functional stuff in Java 8 would probably tempt me back if I was really still in the market for Java work...

bcbradley04:03:29

that probably isn't fair, but thats the reality of it as far as i'm concerned, and C++ is a great technical asset but a poor language

bcbradley04:03:21

For instance, I don't really like object oriented programming and Java is only second to smalltalk in that regard, but it has terrific documentation because it is so widely used

bcbradley04:03:27

so that weighs in its favor

bcbradley04:03:33

even though the language itself isn't that remarkable

bcbradley04:03:43

another thing is the JVM, which is most definitely remarkable

seancorfield04:03:49

I would say that "C++ is a testament to what the ISO/ANSI process can achieve" and that's both a compliment and a criticism 😈

bcbradley04:03:15

i feel like clojure has the best package i've seen so far

bcbradley04:03:19

great community

seancorfield04:03:23

Java is a horrible OO language really 😐

bcbradley04:03:23

decent enough docs

bcbradley04:03:27

fantasitc language design

bcbradley04:03:45

did i mention fantastic language design?

seancorfield04:03:05

Heh, yeah, Clojure is one of the best languages I've ever used for production work 🙂

qqq04:03:07

"java" "fantastic language design" ?

seancorfield04:03:16

He means Clojure.

bcbradley04:03:40

i feel like the community is small enough that people know what is going on

bcbradley04:03:54

and that is very helpful, because it saves people from repeating themselves

bcbradley04:03:45

i don't have any proof but i feel like when there are too many people working on providing assets and libraries for the same language, they end up trying to specialize away from eachother to not have duplication

bcbradley04:03:10

and then you end up with a plethora of barely different libraries, so people try to differentiate further by combining pieces to make frameworks

bcbradley04:03:15

and then you end up with javascript

seancorfield04:03:48

I'm curious, what do you think of Clojure's "web framework" story? (or lack thereof)

qqq04:03:54

imho, for OpenGL, js has better bindings (WEbGL) than java / jogl

bcbradley04:03:12

Honestly i think there needs to be less focus on providing a framework for doing x or y, and more on exposing the protocols being used in a data oriented way

bcbradley04:03:54

i can understand why there is an impetus to have something like ring

bcbradley04:03:15

but what i really would like to see is something more basic

bcbradley04:03:26

that kind of stuff

bcbradley04:03:32

http can be built on top of that

bcbradley04:03:40

no need to piggy back on java all the time

bcbradley04:03:10

the benefit of doing that is that it informs the community about how to work with low level sockets in clojure

bcbradley04:03:18

and that is more important than the technical aspects of the problem

seancorfield04:03:24

Clojure is designed as hosted language -- it is designed to piggyback on the JVM (and leverage its libraries).

bcbradley04:03:48

thats fine, but most of the people i've met who have worked in clojure aren't jvm experts

bcbradley04:03:58

to me its about getting the most value you can for the least cost

bcbradley04:03:10

you can make ring, have a cost X to develop it, and get a value Y to the community

bcbradley04:03:30

or you can make low level internet protocol functions, get a lower cost X, and higher value Y to the community

seancorfield04:03:49

Ring is a great abstraction for HTTP. I don't see any value in dealing with lower level stuff in this day and age.

bcbradley04:03:59

thats precisely my point

bcbradley04:03:05

you don't see any value in it and nobody does

bcbradley04:03:16

because nobody can see it, because its hidden away and has been for a long time

bcbradley04:03:30

let me give an example of how it might be useful

bcbradley04:03:41

tcp orders packets, udp doesn't guarantee order

bcbradley04:03:57

order doesn't matter for set or map key/value pairs, but it does matter for arrays

seancorfield04:03:09

Back in the day I wrote my fair share of SNA-level applications. It's a waste of everyone's time to have to recreate all that again.

qqq04:03:25

brbradley: are you thinking of CRDTs over UDP ?

bcbradley04:03:47

i'm thinking of edn over a low level socket that isn't as basic as udp but isn't really tcp either

bcbradley04:03:02

it only does the work it absolutely has to do

bcbradley04:03:22

it won't order something when order doesn't matter

bcbradley04:03:28

but it will guarantee arrival of everything

bcbradley04:03:40

this is the kind of stuff i meant when i said it would produce value to the community

bcbradley04:03:51

people don't think of this kind of thing in clojure / java communities

bcbradley04:03:01

but they do in plain c (and sometimes c++) communities

bcbradley04:03:27

thats because people don't have the tools to ask these questions or inspect their assumptions

seancorfield04:03:56

I suspect you'll have a hard time convincing anyone in the Clojure community to care about that low-level stuff when Clojure is all about abstractions and higher-level work.

bcbradley04:03:05

i suspect you are right

bcbradley04:03:20

that doesn't mean i'm wrong though

seancorfield04:03:46

I haven't been interested in that low level stuff for decades, since the world standardized on higher level abstractions 🙂

bcbradley04:03:11

if it existed and it was as easy to use but gave you more power, speed, or whatever

bcbradley04:03:13

you'd use it

tbaldridge04:03:22

I can't see what that would get you though, esp considering how large TCP packets can be these days

bcbradley05:03:10

idk you'd have to take a deep look into the IP layer to figure out how and why tcp packets are so large

bcbradley05:03:23

if its for performance, why, and how does it save bandwidth to make packets huge?

bcbradley05:03:43

thats rhetorical btw

bcbradley05:03:09

i'm just pointing out that clojure seems to be missing something

bcbradley05:03:15

its hard to put a name on it

bcbradley05:03:31

it feels like yoda being carried around by luke

bcbradley05:03:37

teaching us the way of the force

tbaldridge05:03:46

Okay, so let's say we're sending tuples of [int, int, string] (as Datomic does), app-level compression is going to give you way more bang for the buck then trying to say that you can re order some tuples

bcbradley05:03:56

but it has to be carried around

bcbradley05:03:03

it doesn't have the agency, and not for the lack of power in the language

bcbradley05:03:09

but the people who use it don't have the agency

bcbradley05:03:28

i understand it is hosted

bcbradley05:03:48

but the people who use it generally don't understand the interop that well, or know the jvm like the back of their hand

bcbradley05:03:51

something is missing

bcbradley05:03:56

a good book maybe

seancorfield05:03:14

I'm glad there are people who care about that low-level stuff so I don't have to. I care some about HTTP as long as I have an abstraction for it. I care very little for WebSockets but I'll use an appropriate abstraction for it. Same with JDBC. I don't want to have to deal with the protocol under the hood. Even Java-level JDBC is painful. I like using clojure.java.jdbc so I don't have to care about that stuff.

tbaldridge05:03:16

I think you assume too much to think that the Clojure community lacks people who understand the low-level inner workings of networking

bcbradley05:03:08

if they do that understanding isn't well represented or exposed to the rest of the community in libraries, and there is an obvious reason for that

bcbradley05:03:21

because being a hosted language can be a blessing and a curse

seancorfield05:03:42

Dealing with the low-level stuff is very much a minority concern tho'.

bcbradley05:03:42

i'll just say this

tbaldridge05:03:43

and I still don't get what all that buys you. I have co-workers who have saturated 1gig connections via Clojure applications.

bcbradley05:03:46

someone has to be first

bcbradley05:03:56

someone has to be the first to implement a solution to a new protocol

bcbradley05:03:04

to interface with a new kind of technology

bcbradley05:03:06

and so forth

bcbradley05:03:12

a small group of people can do it

bcbradley05:03:16

but someone has to be first

tbaldridge05:03:37

If you want high performance you don't have to write your own protocol, you just need a few tweaks here and there.

bcbradley05:03:57

i think you are stuck on the example i gave

bcbradley05:03:00

it was just an example

bcbradley05:03:18

i'm talking about culture, what people think and how they think, and how the community organizes to solve problems

tbaldridge05:03:37

So here's the problem...you aren't really saying anything, besides "I think clojure people don't understand how to do low level stuff...it's a feeling I have"

bcbradley05:03:59

in order to better organize people, you want idiomatic solutions to various problems exposed to eachother in the clojure language

tbaldridge05:03:00

I agree with some stuff, Ring is not the end-all-be-all. It's never been a great solution. That's why we have alternatives.

bcbradley05:03:16

its about having a feedback loop, basically

bcbradley05:03:48

people do best when they can see eachother, and can see what they are working on clearly

bcbradley05:03:14

if all the people who are best at solving problems of a specific domain X, say networking, don't share their knowledge with others

bcbradley05:03:21

it doesn't help anyone else learn or grow

bcbradley05:03:50

when you have a hosted language, if these smart people leverage the host language rather than exposing their knowledge to others in a library

seancorfield05:03:51

Clojure is a very diverse community. But as with (nearly) all language communities, the vast majority of them are working on high-level applications and not low-level wiring stuff. That's true in every technology.

bcbradley05:03:59

it diminishes the effectiveness of the community

tbaldridge05:03:28

that's completely false, just because something is hosted doesn't mean you "can't" get lower.

bcbradley05:03:45

can't and won't are different

bcbradley05:03:53

we are talking about people here

tbaldridge05:03:54

I can do manual memory management in the JVM if I want...there's just never much use for it, except when there is, in which case I reach for a library that helps with that

bcbradley05:03:48

people learn from demonstration, examples, rarely from introspection

tbaldridge05:03:53

But I'd love to hear a concrete example, thus far this discussion is way to abstract

bcbradley05:03:03

i'm sorry then it is an abstract topic

bcbradley05:03:08

it won't get any more concrete than this

seancorfield05:03:27

Then you won't convince anyone -- and I think you have an extremely narrow view on this.

bcbradley05:03:41

then i've failed to make a convincing argument

seancorfield05:03:06

I think you over-estimate the importance of this issue to tech communities at large...?

tbaldridge05:03:43

Yes, it's not good enough to say "there's a problem that I can't describe because it's too abstract". You have to say "X is a perfect example of how the community is harmed by the situation I'm describing"

seancorfield05:03:51

@bcbradley Can you provide an example of a tech community where you think there is this different priority?

bcbradley05:03:09

from my point of view i have described it as best i can, and either the topic is to abstract to communicate briefly in english or I have made an assumption somewhere about our common understanding that isn't actually common, and that is causing a misunderstanding or lack of comprehension

bcbradley05:03:45

in either case its my fault for not being more precise or verbose

bcbradley05:03:19

my basic assumptions are that people have to sense in order to learn

bcbradley05:03:26

new knowledge cannot be generated without new sensation

seancorfield05:03:57

@bcbradley I'll ask again: can you provide an example of a tech community where think there is this different priority?

bcbradley05:03:08

and if there is anything whatsoever that would inhibit people's ability to sense eachother, then twenty eyes drops down to two eyes

bcbradley05:03:29

@seancorfield no, i don't have an example i have principle

seancorfield05:03:08

Well, remember that technology is a tool that people use for solving production problems. It's not some abstract ideal thought experiment.

bcbradley05:03:20

i don't have an example of dark matter either, but it probably exists

tbaldridge05:03:34

Silo'd programmers are a real thing, but in general you'll find Clojure programmers to be very open-minded. For that matter core.async was created after we saw what some other languages were doing and said "heh, we can do that too".

bcbradley05:03:24

then the argument i made about exposing people as much as possible to eachother's expertise should be reasonable

tbaldridge05:03:07

Right, but what I'm saying is we do that, if you have evidence to the contrary I'd love to hear it.

bcbradley05:03:35

its not that it isn't possible for a person who is versed in networking to expose his expertise to others in the community in the clojure language, its just that he has little incentive to do it because he doesn't need to create the library for his own purposes

bcbradley05:03:51

he can just use the hosted language

tbaldridge05:03:57

Right, and if he doesn't, what's the harm?

bcbradley05:03:10

a better question is if he does, whats the gain?

bcbradley05:03:48

it makes the community that much better because the people in it are able to learn from him

tbaldridge05:03:56

The cold hard truth is...at the rate programmers are paid in relation to the cost of hardware, it's simpler/cheaper/etc. to spin up a new sever than to save 1% in processing power by optimizing some sort of network protocol.

bcbradley05:03:10

you are still stuck on that example

bcbradley05:03:17

i shouldn't have gave any example

seancorfield05:03:50

You need to give a concrete example that is compelling. Otherwise it's just words and your vague feeling that something is wrong.

bcbradley05:03:02

everything in english is just words

bcbradley05:03:12

whether they have meaning or not to you is dependent on how hard you think about it

bcbradley05:03:15

examples are easy to think about

bcbradley05:03:20

abstract topics aren't

bcbradley05:03:31

examples are also easy to misinterpret

tbaldridge05:03:34

No that example is fine, it applies to many things. I could learn from a DB admin how to split tables across spindles, but 99% of the time it doesn't matter, and isn't worth the time/cost

bcbradley05:03:39

because they rarely capture the totality of the subject

seancorfield05:03:37

You're hiding behind a claim that the Clojure community don't think -- and that seems a rather insulting position to take without any concrete evidence. Do you understand how high-handed you sound? (that's a genuine question -- I'm curious if you realize how you are coming across here)

bcbradley05:03:54

no i'm sorry i didn't mean to come off as offensive

bcbradley05:03:21

i'm only asking you to think, not to present an argument or counterpoint

bcbradley05:03:27

if i'm wrong, i'm wrong

bcbradley05:03:35

there is probably still value in what i'm saying

bcbradley05:03:41

insofar as it provokes you to think

bcbradley05:03:25

i think the clojure community has members that are great at doing X or Y

bcbradley05:03:41

but they could have more if there was more knowledge sharing

seancorfield05:03:57

But we are thinking. And the community do think. And that's what is so insulting about your position, I'm afraid.

seancorfield05:03:18

"I'm only asking you to think"...

bcbradley05:03:05

Then perhaps we think differently, or have different assumptions so to reach different conclusions

seancorfield05:03:09

With my Clojurians Admin hat on, I'm tempted to ask you to take a time out from the community and think yourself about this sort of engagement...

bcbradley05:03:55

i'm sorry I've offendend you

seancorfield05:03:40

You haven't "offended" me -- you're just taking a very insulting position to the community as a whole, without providing justification for it.

bcbradley05:03:29

if you see it as insulting and I don't, what could cause us to have such different viewpoints?

bcbradley05:03:40

I'm not an expert in clojure or java, and I have far less experience than most of the people in this channel

bcbradley05:03:49

i'm not making an argument from authority

bcbradley05:03:57

i'm not even making an argument

bcbradley05:03:15

i'm just trying to express an observation

bcbradley05:03:27

people do better when they can share with eachother

tbaldridge05:03:30

Clojure programmers think more than any programmer group I've ever been a member of. Infact one of the original talks Rich gave was on the very subject of thinking. Hammock Time was coined by this community, so if you are saying the community doesn't think and can't give a good example or argument to support your opinion, I have to assume you are either uninformed about the community or trolling.

tbaldridge05:03:17

It's fine to say that we don't think...and I'll listen...as long as you can give me an example or argue a point on that subject. Otherwise it's just a waste of everyone's time.

bcbradley05:03:28

i'm not saying the community doesn't think i'm saying they waste thought

bcbradley05:03:41

if two people solve the same problem, thats wasted thought

tbaldridge05:03:49

Great, have an example?

bcbradley05:03:24

nevermind it seems I should probably keep my musings to myself

bcbradley05:03:38

i'm very sorry if i've offended anyone

bcbradley05:03:42

it wasn't my intention

tbaldridge05:03:46

It's not offensive per se, it's just if you want to argue a point, bring some facts, or some common ground we can meet on.

tbaldridge05:03:46

And it's very rare that two people will solve the same problem, often there are tradeoffs. There is no such thing as "the best solution" in programming, everything is a tradeoff, everything is a compromise. So often these duplicated projects are just a different sent of tradeoffs valued over other sets of tradeoffs.

shaun-mahood05:03:08

As an anecdote after reading through the conversation above, I have seen a much greater focus on low level performance (for a purpose) and interest in the implementation details of underlying libraries in the Clojure community than I ever did in the .NET community. I spent much of my University days working in C and assembly, and that is still the way I think about problems first, and I really could never bring myself to care about low level networking details even when working at that level - it simply doesn't matter to the problems I want to solve. In every area that is relevant to my work, the Clojure community has been a source of inspiration and information that has directly helped me solve more problems than I ever have before, and has led to me being more interested in the fundamentals of my work than ever before. I'm now going back and re-learning CS fundamentals that seemed pointless until the Clojure community showed me why they mattered in a practical way.

bcbradley05:03:18

You shouldn't take my rambling as a representation of the community

bcbradley05:03:57

but i'm happy you are enthusiastic about clojure

shaun-mahood05:03:14

@bcbradley: I'm sure it's just something lost in translation - there are a lot of interesting things happening in Clojure, but it can be a lot to take in to really see the breadth and depth of what people are working on. Onyx and Jepsen are somewhat unique (at least from what I can see), ClojureScript is pushing a ton of awesome boundaries, Will Byrd said at the last Conj that the clojure community is the main place his research has found acceptance, Dragan Djuric is doing interesting work in high performance computing, and more and more. Hang out for a while and spend some time seeing what is happening and you may get a better sense of the community and where things are moving - maybe you'll find something that you can contribute to that will bring us closer to the ideal you are looking for - at the very least you should be able to better understand and verbalize the root of what you were trying to get to above.

tolitius06:03:21

@bcbradley which language do you feel most comfortable with? I believe you are "coming hot" to something that's new to you and you want to map patterns and ways you used to program to Clojure. You hope to find concrete examples, but you can't. If I am at least a little bit correct in my assumptions, I would recommend not to quit, and give Clojure a chance. As with anything new, you'll need time. One of the reasons you can't find examples could be because the way you know how to program could be very different in Clojure. The hardest part here is to ask the right questions and it just needs time and practice.

tolitius06:03:06

@bcbradley I agree with you that it is great to understand how TCP/IP works, for example, for two reasons: it is in fact needed for some problems (i.e. HFT) + it is just interesting to look under the elephants that hold the Earth, cause maybe the Earth is oblate spheroid and there is no turtle 🙂

bcbradley06:03:48

@tolitius it isn't that, I just have some beliefs about what kind of environment people work best in and failed to communicate it

bcbradley06:03:33

@shaun-mahood I'm sure I'll be able to communicate it better one day

martinklepsch10:03:01

I never noticed that (merge #{:a} #{:a}) returns #{:a #{:a}} 😮

bronsa10:03:30

merge is only defined for maps and boils down to reduce conj so it's easy to see why that happens

bronsa10:03:59

but yeah it's a bit odd, agreed one would expect merge to work as union on sets

istvan10:03:32

martinklepsch (union s1 s2)?

istvan10:03:18

did not know that merge works on sets 🙂

martinklepsch10:03:33

into also works fine. I was just surprised 🙂

dergutemoritz10:03:05

Tidbit about into for sets: The other day I wondered why one would want to use clojure.set/union instead of into and found that it's smart about conjing the smaller set into the larger set. Could be useful sometimes 🙂

dergutemoritz10:03:17

Then again, it's an implementation detail. Oh, another reason would be to unionize more than two sets!

danp12:03:06

Hey all, I'm new here and looking for some help with some Java interop.

danp12:03:04

Is this the right channel for asking for such? 🙂

pesterhazy12:03:02

ask away

danp12:03:02

Excellent. Basically, I'm trying to leverage the Java SDK of Oracle Data Integrator, and being fairly new to Clojure, it's the first bit of interop I've tried.

danp12:03:37

Created a lein app, put a symlink in the .../resources folder that points to the ODI jars folder and then referenced the jars as I've needed them in the :resource-paths of my project.clj.

pesterhazy12:03:02

if you want to access code, shouldn't that be in a source (not resource) folder?

danp12:03:47

I've managed to create some connection information objects, but when I try and create an actual connection I'm getting a NoClassDefFoundError, despite the class required being in the jar.

danp12:03:00

Oh, I'm not sure - I just googled "how to use local jars leiningen", or similar. It seems to work up to a point.

pesterhazy12:03:09

(ignore my previous comment, not sure that was relevant)

danp12:03:21

no worries 🙂

pesterhazy12:03:35

what's the exception you're seeing?

danp12:03:42

The error is CompilerException java.lang.NoClassDefFoundError: could not initialize class oracle.odi.core.DataSourceManager$1, compiling:(core.clj....

danp12:03:21

In the :import, I've recently added DataSourceManager$1 to try and fix the error, but to no avail.

pesterhazy12:03:32

have you tried importing just DataSourceManager?

danp12:03:44

Same result 😞

pesterhazy12:03:13

I think the $ in the class name means that it's an inner class

pesterhazy12:03:16

pretty sure that you need to add something to the classpath

pesterhazy12:03:50

you could try taking leiningen out of the equation as an experiment

danp12:03:22

Okay, I'll give it a go, but not sure where to start.

pesterhazy12:03:45

java -cp clojure.jar:odi-jar.jar clojure.main

pesterhazy12:03:06

then you can import classes from the repl

danp12:03:40

ok, so will need all of the jars I've got in my project.clj I'm guessing

pesterhazy12:03:58

just try minimal example

pesterhazy12:03:30

that only creates a connection. That shouldn't require too many other dependencies, right?

danp12:03:12

There's a handful it needs - jdbc, spring ones, etc - but they're not extensive

danp12:03:16

give me a mo

pesterhazy12:03:15

you can also run lein classpath

pesterhazy12:03:22

should give you a good place to start

danp12:03:37

That's helpful 🙂

danp12:03:23

Ok, I've got a REPL

danp12:03:53

I'm just going to add the necessary jars to the classpath and will see how I get on

danp12:03:30

Interesting - I've just ran the code snippet above, and this time it's complaining about an Apache logging lib not being present.

danp12:03:46

Should be in my lib folder though so will add it to the cp.

danp13:03:02

There seem to be a few libs it's expecting that aren't in the cp - I'll go through them and add them one by one as needed and get back when I've exhausted them!

danp13:03:44

Is it usual to have to go outside lein?

pesterhazy13:03:12

don't know if it is

pesterhazy13:03:38

I usually work with boot, not lein so maybe someone with more experience with leiningen can chime in

pesterhazy13:03:50

I suggested trying the pure clojure jar to isolate the issue and to find a minimal example

pesterhazy13:03:14

there could be an easier approach

danp13:03:44

No, I appreciate it - I'm currently doing the Brave Clojure book, so my knowledge and it accumulation has been pretty linear. This stuff is really helpful!

danp13:03:23

It's actually run without error now, but when I do (type odi-instance) I get another error, seemingly another missing library!

danp13:03:39

And now I've managed to get through to an actual Oracle error!

jooivind13:03:33

anyone using docjure here?

manutter5113:03:55

I’ve used the spreadsheet read/write stuff in docjure

jooivind13:03:10

Cool. Do you know of any way to set columns to autoscale to content?

manutter5113:03:16

not offhand. Have you looked at the Apache POI docs?

jooivind13:03:36

Will check them out

danp13:03:51

@pesterhazy, I've got it (mostly) working with Leiningen now. Thanks for the guidance 🙂

pesterhazy13:03:48

Out of curiosity, what turned out to be the problem?

danp13:03:42

I have no idea, when I did everything outside lein, it was pretty clear about what was missing. When I had a "working" classpath, I just copied the relevant jars into :resource-paths.

pesterhazy13:03:35

sometimes when you try something in the repl, you get a partial exception. In that case you can type (pst) to get more complete information

danp14:03:53

Great - I'll make note of that one. Thanks again.

manutter5114:03:32

@jooivind It doesn’t look like docjure supports auto-size columns directly, but you should be about to call the autoSizeColumn method on the worksheet directly, via Java interop. I found a java example at http://www.programcreek.com/java-api-examples/index.php?class=org.apache.poi.hssf.usermodel.HSSFSheet&amp;method=autoSizeColumn

plins14:03:00

hello everyone, im using compojure-api, and Im having doubts about de-structuring

(POST "/endpoint" []
       :summary "info about endpoint"
       :return {:responseField s/Num}
       :body [my-entity
              {:field1 s/Str
               :field2 s/Str
               :field3 s/Str}]
        (some-func (:field1 my-entity)  (:field2 my-entity) (:field3 my-entity)))
is there a way to avoid using the getters like (:field1 my-entity) (:field2 my-entity) (:field3 my-entity) ?

juhoteperi14:03:14

@plins Yes, either using normal destructuring: :body [{:keys [field1 field2 field3]} {:field1 s/Str :field2 s/Str :field2 s/Str}] or using Plumbing letk syntax which allows the same without repeating field names: :body-params [field1 :- s/Str, field2 :- s/Str, field3 :- s/Str]

plins14:03:28

the second option seems way more attractive, i will take a look into it thx!!!!

jooivind14:03:33

@manutter51 i got it working by accessing the native java method, thanks!

sveri14:03:31

Hi, I have a problem where I want to combine two lists in a specific way. So lets say I have a fixed list of privileges: (def privileges [:none :deny :grant]) and a list of rights: (def rights [:save-table :save-tree]) Every right can be set one privilege. So for one right the result would look like this: [{:save-table :none} {:save-table :deny} {:save-table :grant}] And now comes the tricky part. For every additional right the rights can be combined with the privileges in every possible way. So for two rights the result would look like this:

[{:save-table :none :save-tree :none}
 {:save-table :none :save-tree :deny}
 {:save-table :none :save-tree :grant}
 {:save-table :deny :save-tree :none}
 {:save-table :deny :save-tree :deny}
 {:save-table :deny :save-tree :grant}
 {:save-table :grant :save-tree :none}
 {:save-table :grant :save-tree :deny}
 {:save-table :grant :save-tree :grant}]
I played around with permutations and tried some stuff using reduce and map, but just cannot find a good solution. Any ideas?

rauh14:03:05

@sveri Outer product is done by for usually.

tap15:03:37

(mapv #(zipmap rights %) (for [p1 privileges p2 privileges] [p1 p2]))

sveri15:03:10

@rauh @tap Thanks, I totally forgot about for. Is there a way to use it with a dynamically sized list?

rauh15:03:34

@sveri a 3-way for is the same as a 2x2-way for (where the second for uses the result of the first for). So you can just iterated with whatever construct (loop(?)) and use a 2-way for until you're done. Did that make sense? 🙂

sveri15:03:14

@rauh Conceptually yes, gotta see if I can translate it into clojure code 🙂

rauh15:03:31

@sveri why not use math.combinatorics's cartesion product?

sveri15:03:07

@rauh I have not found a way to make it work, although, I mostly tried my way with permutations.

rauh15:03:18

@sveri Yeah, what you want, isn't really a permutation. But a combination.

rauh15:03:33

aka outer product, aka cartesian product.

sveri15:03:15

I managed to to use the cartesian product to build the privileges, but its the same problem. It takes several seqs as arguments, which is dependend on the number of rights in the rights vec: (comb/cartesian-product privileges privileges)

rauh15:03:15

@sveri I don't understand. -v

sveri15:03:48

This is what I came up with for a dynamically sized rights vec:

(loop [counter (dec (count rights)) privs [:none :deny :grant]]
  (if (zero? counter)
    privs
    (recur (dec counter) (for [p1 privileges priv1 privs] (vec (flatten [p1 priv1]))))))

rauh15:03:00

(let [priv [:none :deny :grant]
      rights [:save-table :save-tree]]
  (apply cartesian-product [rights] (repeat (count rights) priv)))

sveri15:03:01

Together witz zipmap it does the job 🙂

rauh15:03:22

Well that looks good too 🙂

sveri15:03:21

Yea, thanks to you both

plins17:03:08

is it possible to use type hints with defrecord defined types?

tanzoniteblack17:03:29

@plins what's your use case for wanting to?

netsu17:03:02

Is there a way to refer java library from clojure

plins17:03:19

(ns bla.bla
  (defrecord Benefit [^String cpf
                                              ^String username
                                             ^String email
                                              .....]))

(defn my-func [^Double some-field ^bla/Benefit benefit] ....)

netsu17:03:24

I mean without ugly prefix like java.util.bla.bla.bla, is it possible to use something like :as in import like in require? Can't find it in docs, sorry.

pyr17:03:27

say you had a nice corpus of uniform records stored in EDN

pyr17:03:41

which datalog-inspired library would you reach out for to query it?

tanzoniteblack17:03:52

@netsu instead of (:import [java.util.bla]), do (:import [java.util bla]) (note the space there at the end), now in that ns you can refer to it simply as bla

pyr17:03:11

for the workflow: load-edn, build in-memory database, run queries

tanzoniteblack17:03:50

@plins generally type hinting is done to help the compiler make more efficient code by avoiding reflection, but if you're using a record created via defrecord, I'm not sure how you're writing your code that it might be doing reflection to have to figure out what class it's calling things with?

pyr17:03:02

I feel like core.logic is a bit over the top for this, but maybe i'm wrong, I just don't have too much experience with it

plins17:03:51

@tanzoniteblack , right now im just annotating stuff for myself, afterwards i intend to switch to typed clojure

tanzoniteblack18:03:23

if you're just annotating stuff for yourself, either start with typed clojure, or prismatic's schema, or if you're using the 1.9 alpha use spec?

tanzoniteblack18:03:48

@pyr as long as you're planning on loading it into memory anyways, depending on the complexity of your "query", you probably just want to use filter/`remove` to select your items. If the "queries" are rather complex, you might look into selecting them via https://github.com/nathanmarz/specter ?

hiredman18:03:12

pyr: I usually end up building an index using clojure.set/index and querying via get and for

pyr18:03:50

hiredman: that sounds like an approach that would work for me, thanks, i'll look into that

netsu18:03:20

@tanzoniteblack could you suggest how use it inside (ns? Just

(ns local-scope-name
  (:import [java.util.regex Pattern])
not working in expected way.

tanzoniteblack18:03:13

netsu: How are you using it in your code, so I know "what the expected way" is?

netsu18:03:38

with (import [java.util.regex Pattern]) it in scope. But when I simple do (:import [java.util.regex Pattern]) in (ns, it invisible.

tanzoniteblack18:03:44

can you please show me how you're calling Pattern in the code?

tanzoniteblack18:03:51

not in the import

tanzoniteblack18:03:03

but where you're actually trying to use the Pattern class for something

netsu18:03:24

Currently code not working yet, so not sure about corectness. Here example:

(def dribbble (Pattern. "host(); path(shots/?id); queryparam(offset=?offset);"))

tanzoniteblack18:03:46

1.) http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html doesn't have a constructor that just takes a String (which is what you're doing with (Pattern. "host...")

tanzoniteblack18:03:56

you have to use Pattern/compile

tanzoniteblack18:03:13

and 2.) Clojure has a built in reader for regexes; just use #"host..."

netsu18:03:25

I'm all ears.. Ok, sounds reasonable.

tanzoniteblack18:03:33

which avoids having to import Pattern at all 🙂

netsu18:03:18

but now I curios also about why I can't refer to java in :import also... Clojure is still pretty new for me yet.

netsu18:03:05

didn't know about #"", it looks awesome, thanks!

tanzoniteblack18:03:31

(ns data-playground.test
  (:import [java.util.regex Pattern]))

(def test-pattern (Pattern/compile "string"))

tanzoniteblack18:03:53

that code works for me to define an ns, import a Java class by name, and then refer to it later on

tanzoniteblack18:03:16

(again, you should use the #"" reader macro instead of Pattern/compile in Clojure code, but for the sake of example)

ghadi18:03:04

If you're building a regex dynamically, use re-pattern otherwise use the literal

netsu18:03:18

thanks a lot for advises!

pyr18:03:15

tanzoniteblack: I'm a bit beyond the filter / remove phase

tbaldridge18:03:36

@pyr Datomic works just fine for this, the in-memory connection works great, and the query engine is top notch

tbaldridge18:03:15

Odin will auto-index for you and saves you having to think about schemas: https://github.com/halgari/odin

pyr18:03:07

I didn't know datomic had published the free bit on clojars

arthur18:03:10

Good morning folks, I think I'd "doing it wrong" when i type things like this:

(filter #(select [ALL :thing  #{2 22}] %) list-of-maps)

arthur18:03:07

If I want to select members from a sequence of nested blobs based on fancy criteria

tbaldridge18:03:21

Well the fact that you used the words "sequence of nested blobs" says that this data is pretty unstructured. If you can fix that, accessing the data is often easier.

tanzoniteblack18:03:04

@arthur 🙂 you could do the whole thing in specter without the filter, right? Just do (select [ALL ALL :thing #{2 22}] list-of-maps)? So not sure if your "wrong" feeling is because it's an inefficient way to query the data (by looping through the entire list of maps for each individual query you have), or because you're mixing filter and specter?

tanzoniteblack18:03:41

to me, it greatly depends for the speed thing. If you just need to get one or two things out of existing data you already have, it's probably more efficient to just do the specter or filter method. If you have a few hundred things you need to get, then you should probably make your data structure into something more efficient to query (like something with some kind of index). Also important: When/where is this data being run? In production code on a web api? Then you should be using an actual database, not a file. In some kind of one off script? Then who really cares as long as it finishes?

arthur18:03:19

It's actually your code 😉 I'm just fixing it

tanzoniteblack18:03:53

lmao; fixing my old code from when Kurman and I were playing with specter originally then?

arthur18:03:03

I'm also trying to learn specter.

arthur18:03:22

in this case there is a list of maps

arthur18:03:37

and I want to select some of the maps from that list

tbaldridge18:03:57

" In production code on a web api? Then you should be using an actual database, not a file. In some kind of one off script? Then who really cares as long as it finishes?"

tbaldridge18:03:01

Incidentally that's my biggest rational for not using Specter.

arthur18:03:42

(select [ALL ...] ...) returns only the portion being used to do the selection. so in the abve code it returns 22 not the map that under that key had the value 22

nathanmarz18:03:17

@arthur I think you're looking for selected?

nathanmarz18:03:00

(select [ALL (selected? <subpath>)] list-of-maps)

nathanmarz18:03:21

if <subpath> selects anything, then that element will be in the output vector

arthur19:03:40

yes @nathanmarz that's the concept I was looking for

arthur19:03:47

the idea of selectedness

shader19:03:52

is there something like assoc for adding default values only if the field isn't already set?

tbaldridge19:03:00

@shader fnil works for this

tbaldridge19:03:17

or merge is good too

jr19:03:21

(merge default-vals map)

apbleonard19:03:30

Hi - I'd like to take a value from a channel and dispatch it to one of a pool of channels, where the dispatcher picks the channel based on the value. The idea is to process incoming requests concurrently, but make sure that e.g. a given users requests are placed in the same queue so they are handled sequentially.

hiredman19:03:41

have you seen pub/sub in core.async?

apbleonard19:03:09

no ... reading up now :)

apbleonard19:03:16

not sure that would fit...

hiredman19:03:07

you create a pub with a topic function something like (mod (hash whatever) 6) then you spin up go loops that subscribe to topics 0 through 5

dominicm19:03:23

Potentially Seeing some large GC pauses when processing large lazy sequences. How can I start debugging that? Any common gotchas to look out for? The processing includes enriching the sequence from datomic and doing a http post.

tbaldridge20:03:15

@dominicm honestly, use transducers.

tbaldridge20:03:36

I used to have tons of issues with GC and lazy seqs, it's next to impossible to get that to happen with transducers

octahedrion20:03:50

i have a project that's got a dependency on org.clojure1.3.0-alpha2, which isn't accessible - how do I find out which dependency (using leiningen) it is ?

rauh20:03:23

@octo221 Good old lein deps :tree

tbaldridge20:03:31

@octo221 look at lein deps :tree

octahedrion20:03:09

yeh I did that, but it didn't show me where the 1.3.0-alpha2 was

mccraigmccraig20:03:03

did you get an error @octo221 ?

octahedrion20:03:50

yes, when I do lein deps (or lein deps :tree) I get Could not transfer artifact org.clojure:clojure:pom:1.3.0-alpha2 from/to clojure

octahedrion20:03:17

(I'm using clojure 1.9.0)

octahedrion20:03:09

- sorry I should have said it's fine on my local machine - I'm getting the error on a remote clone

mccraigmccraig20:03:26

oh, can you not see where the dep is coming from on your local machine then ? alternatively, put an :exclusions [org.clojure/clojure] at the top-level of your lein project, and include the version you want explicitly in your :dependencies

hiredman20:03:49

clojure in that error message refers to a maven repo

hiredman20:03:17

maven repos can be configured in lein profiles or project.clj or settings.xml in ~/.m2/

hiredman20:03:53

if you are are setup to only use a maven repo you named "clojure", and that maven repo doesn't have the artifact you are looking for, the error would look like that

octahedrion20:03:24

no I can't see where that dep comes from on my local machine with lein deps :tree

hiredman20:03:26

oh, and I bet you are using a really old version of lein too

hiredman20:03:32

(on the remote machine)

octahedrion20:03:54

that could be it....

hiredman20:03:14

at some point, many years ago, the clojure artifacts had their own maven repo (not in central) and I bet lein defaulted to including that, and called it "clojure"

hiredman20:03:42

on the failing machine?

Alex Miller (Clojure team)20:03:45

yes, that was a file system on http://build.clojure.org, which no longer exists (all those artifacts have been moved to maven central)

hiredman20:03:06

lein show-profiles on the failing machine, then look at each profile to see if it specs a repo somewhere

hiredman20:03:55

how did you check the lein version on the failing machine?

hiredman20:03:34

check lein version

octahedrion20:03:21

Leiningen 2.7.1

apbleonard20:03:47

@hiredman Love it. Thank you. Would one care about "consistent-hashing" here...?

hiredman20:03:26

no, hash + mod or some other way to map an open set of values to a closed set is enough, consistent hashing is a whole other thing that comes up in key value stores limiting the shuffle of data around when nodes are added or removed from the cluster

octahedrion20:03:26

got it - it was Incanter - found by repeatedly deleting half the deps from the deps vector!! binary search

tbaldridge20:03:13

ah yes, incanter, the project that hasn't been updated in 3 years. Its surprising how often it turns up.

octahedrion20:03:22

:exclusions [org.clojure/clojure] fixed that one

octahedrion20:03:49

it's a shame about Incanter. I really want that project to go somewhere

octahedrion20:03:23

and when you think people are using R

octahedrion20:03:42

Clojure would be such a superior alternative

shader21:03:17

is there a better way to factor that?

shader21:03:01

the trick is handling rest args vs a pre-collected list of parameters

octahedrion21:03:27

off to sleep - thanks everyone for your help!

kanwei21:03:25

Anyone know a quick way to do pmap with x workers instead of arbitrary workers?

ghadi21:03:42

use core.async's pipeline

shader21:03:37

is there a function that wraps a non-seq object in a list or vec, but leaves a seq alone?

joshjones21:03:37

@kanwei I would agree with @ghadi:

(defn worker [n]
  ; showing a blocking call here.  use pipeline-blocking if your
  ; worker will block. This is for demo purposes only, to simulate
  ; real work
  (Thread/sleep 1000)
  (inc n))

(let [out (chan)
      in (a/to-chan (range 8))]
  (a/pipeline 8 out (map worker) in)
  (<!! (a/into [] out)))

joshjones21:03:07

(the sleep to simulate some real work) -- change 8 above to 1 to observe the parallelism

hiredman21:03:38

pipeline-blocking

joshjones21:03:51

Not what I'm using it for

joshjones21:03:03

I'm just showing how something can take some time.

joshjones21:03:34

Make it a loop that iterates to a billion if you want, same result for the example

joshjones21:03:25

I was sure someone would say that, but hoped it would be obvious in the context that it is an example to demonstrate something taking time.

hiredman21:03:18

the question isn't if it is obvious for us, the question is if it is obvious to people who may not be familiar with core.async and isn't aware that there are multiple kinds of pipelines. You shouldn't just run with an example of pipeline usage you see, there are variants of pipeline that behave differently and you should match the variant of pipeline to the kind of task

joshjones21:03:30

changed the example above with a comment

johnnyillinois22:03:01

Anyone know any idiomatic way to create a map from a keys list and a values list

dpsutton22:03:05

user=> (zipmap [:a :b :c :d :e] [1 2 3 4 5])
{:a 1, :b 2, :c 3, :d 4, :e 5}

dpsutton22:03:09

is that what you want?

johnnyillinois22:03:22

I didn't know about that

johnnyillinois22:03:50

sure beats: (apply hash-map (map #(list %1 %2) keys values))

johnnyillinois22:03:52

Does someone know how to set the ns lookup path in the repl? Suppose I have a file in src/clj/app_name/core.clj The namespace is app-name.core In the repl, I can run: (use 'clj.app-name.core) (use 'app-name.core) I would like to do this with a single command.

hiredman22:03:55

app-name needs to be app_name

hiredman22:03:41

for reasons no one really cares about, dashes become underscores when mapping namespaces to files

hiredman22:03:09

what is the namespace defined in the file?

johnnyillinois22:03:35

thanks @hiredman but that is not the problem

hiredman22:03:59

there isn't a "path" you change, file names match the namespace name defined in the file, and clojure loads files from the classpath when you require or use the namespace

johnnyillinois22:03:00

The repl base ns lookup path needs to be prefixed with clj

johnnyillinois22:03:45

Can you explain more what you mean previously?

hiredman22:03:47

there isn't a "base ns lookup path"

hiredman22:03:07

there is the classpath and classloaders

hiredman22:03:22

the classpath is a thing some subset of classloaders care about

hiredman22:03:41

classloaders are a common way to load code an resources on the jvm

johnnyillinois22:03:41

So can I prefix the classpath?

hiredman22:03:08

presumably you are using some kind of tooling and not just a bare clojure repl

hiredman22:03:19

that tooling sets up the classpath when it launches the repl

johnnyillinois22:03:24

The build step is customized

johnnyillinois22:03:55

We are building an SPA with cljs but our acceptance tests run on clojure

hiredman22:03:53

in the configuration of whatever build tool you are using, you are putting src/ on the classpath (or the build tool just defaults to it) but you want to put src/clj

johnnyillinois22:03:59

the project.clj is configured so lein knows to use src/clj as the base source-path for the clj build and src/cljs as the base source-path for cljs build

hiredman22:03:22

have you restarted your repl since then?

johnnyillinois22:03:17

Not sure what you mean?

johnnyillinois22:03:38

I have used multiple instances of the repl

hiredman22:03:13

lein figures out the classpath once when it start the repl, if you make changes to project.clj you have to start new repl processes to see them

hiredman22:03:44

what is the exact error you see when you require the namespace?

johnnyillinois22:03:35

CompilerException java.io.FileNotFoundException: Could not locate acceptance_test/utils__init.class or acceptance_test/utils.clj on classpath.

johnnyillinois22:03:09

(use 'acceptance_test.utils)

hiredman22:03:23

no underscore in the namespace name

hiredman22:03:32

underscore in the file name, dash in the namespace name

johnnyillinois22:03:39

that was a typo

johnnyillinois22:03:59

The problem is the repl is not using the same source-path as the build step

johnnyillinois22:03:48

Anyways, I got it to work. Thank you for your time

ejelome23:03:16

sorry to post this here but #testing channel seems to be not so active: https://clojurians.slack.com/archives/testing/p1489016676000099