Fork me on GitHub
#clojure
<
2017-12-06
>
mikepence00:12:23

hello again, all. been a while. I am back in a ruby shop, where I get quizzical looks when I say that a handful of functions can do what OOP does, but simpler, easier, and faster. how best to illuminate my co-workers?

phronmophobic00:12:26

I’ve found that I’ve grown more convincing as I’ve become better at explaining my thoughts. developing the right vocabulary for describing design tradeoffs has been a huge help. ie. rather than just saying something is better or worse, being able to describe design tradeoffs in terms of coupling, cohesion, or other code attributes. also being able to explain how a design makes tradeoffs in terms of quantifiable metrics can also help (eg. how a design makes tradeoffs in terms of latency vs throughput)

phronmophobic00:12:05

I think the first book that helped me be more descriptive about how to describe the tradeoffs of different designs was code complete

phronmophobic00:12:59

with respect to ruby, rich hickey did give the key note at a rails conf, https://www.youtube.com/watch?v=rI8tNMsozo0

phronmophobic00:12:32

here’s a great list of different code attributes (with definitions!)

caleb.macdonaldblack00:12:29

Any quick little way to convert [["a" "b"] ["c" "d"] ["e" "f"]] to [["a" "c" "e"] ["b" "d" "f"]]?

noisesmith00:12:40

@caleb.macdonaldblack (map vector c)

> (apply map vector [["a" "b"] ["c" "d"] ["e" "f"]])
(["a" "c" "e"] ["b" "d" "f"])
(apply here because it was all in one vec)

caleb.macdonaldblack00:12:38

That’s the one. Thanks guys

noisesmith00:12:58

(def transpose (partial apply map vector)) is a remarkable little chunk of code (I learned it on IRC I forget who from) (edit: better name)

caleb.macdonaldblack00:12:46

Oh thats neat. Thank you!

rainmote02:12:09

Have better debugging tools? I need parse multi-layer json object, it’s very easy wrong, I use Idea IntelliJ tool.

seancorfield03:12:37

@rainmote Can you be a bit more specific about what you're stuck on? Are you using Cheshire to parse the JSON?

rainmote12:12:14

I need process it, eg:add avg etc... Probably because I have just started. Thanks for you’re help, I have solved the problem, and I known the Cheshire project.👍

vemv04:12:10

what's form?

vemv04:12:39

in the source I can see it serves some internal Specize protocol. still confused

Alex Miller (Clojure team)04:12:24

it’s a literal form to use when reporting failures in lieu of the normal form of the spec

Alex Miller (Clojure team)04:12:49

typically would not be something you would use as a user (which is why it’s not doc’ed I think)

Alex Miller (Clojure team)04:12:33

I think it may actually be vestigial at this point

Alex Miller (Clojure team)04:12:45

I don’t think it’s either called or does anything useful now?

vemv04:12:15

thanks for the input @alexmiller! funny, I was implementing something similar right now, a check-with-explanation-value helper

vemv04:12:44

use case: {:post [(check-with-explanation-value some? % coll)]}

vemv04:12:27

because some? nil cannot give useful information, so we use coll (a defn argument) as the explanation value

Alex Miller (Clojure team)04:12:10

there’s a ticket with a patch for something like that if it’s useful to you

Alex Miller (Clojure team)04:12:54

actually, I guess it isn’t what you want, nvm

New To Clojure06:12:46

@rainmote >Have better debugging tools? I need parse multi-layer json object, it’s very easy wrong, I use Idea IntelliJ tool. Why current Intellij tools are not enough? I use CIDER debug and it's good enough so far (it shows values of all expressions starting from inside ones).

rainmote02:12:58

I send message lost yesterday:cry:. I’m beginner and I just start use idea tool, thanks for your help, I will learn it.

cmal08:12:53

Hi, I have a problem with my clojure backend project. When my project starts jetty server, the whole nginx on this linux, including static resources which are out of the clojure project, starts to slows down for accessing. It shows a long orange Initial connection time in Chrome. What can this kind of problem be? Thanks!

fmind08:12:33

@cmal do you see some high resource usage on your system (CPU, RAM, IO) ?

cmal08:12:33

I used # netstat -anpl | wc -l to see tcp connections. some 300-500 counts.

cmal08:12:48

# uptime
 16:36:34 up 474 days,  1:13,  7 users,  load average: 2.71, 1.92, 2.17

cmal08:12:56

and something like this:

# top
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
11450 works     20   0 1627m 498m  14m R 98.9  3.1   2:14.78 grunt
 4035 works     20   0 6748m 1.7g  16m S 88.3 11.2  13:44.59 java
but it seems natural in my server. we have 4 cpu cores.

fmind08:12:50

is is a shared or a dedicated server ?

fmind08:12:19

can you run sudo iotop -a on your server ?

fmind08:12:03

maybe the two processes compete for the IO resources, otherwise I don't see why they would slow down each other

fmind08:12:49

you can also do a curl from the server (localhost) to see if it is slow as well

cmal08:12:02

we run ssh between cloud servers and it is fast. Only when we do remote ssh, or access https service in browsers, it is slow.

cmal08:12:46

and if we stop the clojure(java) process that runs jetty, it recovers.

cmal08:12:48

Does that mean jetty do a lot of io that slows down the server?

fmind08:12:15

it can be jetty or the JVM. One of them can take too much memory/IO and exceed your cloud provider quotas

fmind08:12:58

try to use an alternative to see if it changes something: https://github.com/ztellman/aleph http://www.http-kit.org/

fmind08:12:13

if you use ring/compojure, they can be used as drop-out replacements

cmal09:12:37

@fmind Thank you very much.

kgofhedgehogs11:12:32

In my project ring-middleware-format:jar:0.7.2 imports org.clojure:tools.reader:jar:1.0.5 And immutant:jar:2.1.9's dependency tries to import org.clojure:tools.reader:jar:0.10.0 which is omitted for conflict What to do in such situation? Can bad things happen due to omit of this dependency?

dominicm11:12:05

@kgofhedgehogs There's very little you can do. The bad thing that can happen, will only happen is tools.reader changes it's API between 0.10 and 1.0.5. Rich has firmly committed to not breaking APIs, ever. And I suspect org.clojure/tools.reader will follow this pattern.

kgofhedgehogs11:12:27

Okay, also. In project I have dependency [clj-time "0.14.2"] and ring:ring:jar:1.6.3 want clj-time:clj-time:jar:0.11.0. what to do here? I should use clj-time 0.11.0?

dominicm12:12:19

@kgofhedgehogs I personally tend to use latest as much as possible. I've not had an issue doing this.

mccraigmccraig12:12:26

@kgofhedgehogs whatever works - i tend to try the later dependency first and exclude the earlier on the presumption that the later version will have bugfixes, but occasionally you have to do it the other way around and occasionally you are stuffed and have to go make PRs for some of your dependencies

kgofhedgehogs12:12:49

But I think, what there maybe api changes between 0.14.2 and 0.11.0 versions.

kgofhedgehogs12:12:46

Here is clj-time changelog https://github.com/clj-time/clj-time/blob/master/ChangeLog.md I didnt see any api change mentions

kgofhedgehogs12:12:58

So Ill use 0.14.2

pbaille12:12:13

hello, does anybody here is aware of a currently maintained libray like prismatic.plumbing (specially its graph related facilities) ?

pbaille12:12:12

@fmind thank you! i will look at those

markus13:12:54

Can anyone recommend a Clojure library for calling into an external C library?

Alex Miller (Clojure team)14:12:02

Clojure can use Java JNI to call native code

Drew Verlee14:12:37

So there are quite a few a ways to approach writing more reliable, evolvable software. * specifications: schema, spec * doc enhancers like literate programming (marginalia) and interactive viewers (klipse, Dynadoc) I think each has a set of trade offs. Has anyone written about this subject at a meta level. I think spec does a great job of allowing a user to add more reliability as it becomes more important. But i think growing software has to do with building a human narrative about a system, which, i think is worth creating non code documentation to create. I would like to see a tool which creates a narrative about a system, and can leverage existing ways of communicating: spec, unit tests, doc strings… and also grow and be viewed in different ways as the system evolves. Does anyone know of anyone interested in this, or any works on this subject?

tbaldridge15:12:13

@markus.gustavsson I'd recommend JNA, it's super seamless with Clojure

tbaldridge15:12:55

(.invoke (Library/getFunction "libc" "sin") [0.42])

tbaldridge15:12:06

That's about the extent of the psudeocode you need with JNA

mpenet15:12:01

jni is much faster tho (last time I used it, few years back)

tbaldridge15:12:39

@mpenet define "much faster" 🙂

mpenet15:12:00

in my case it was an order of magnitude faster

mpenet15:12:41

I was interacting with a lib related to rf cavity simulation, so quite fast internally

tbaldridge15:12:19

As always the problem is: * JNA gives you almost instant access to C libs a a slight perf cost (we're talking about 10x slower over the cost of a function call, that's not much) * JNI gives you near native function call speeds, but you have to write/generate/maintain a wrapper and all the associated build code

mpenet15:12:21

but jni requires more ceremony for sure, I had to write some java on the side and then call from clojure

mpenet15:12:37

exactly as you said

mpenet15:12:50

not sure what you mean about the build code, a simple lein file was good enough, I had the java wrapper and the clojure code living in the same project, with the .dll (windows... sigh) in question (I guess it works the same with .so files)

markus15:12:09

@alexmiller Thank you. I was hoping for a more painless way though as I am going to use a C/C++ library with a lot of functions. I already started wrapping it in C, Java and Clojure but it is not much fun. @tbaldridge Thank you for the suggestion. I guess that is as good as it gets right now in the general case. Could not find any maintained pure Clojure libraries. I am actually the author of clj-native and the bits in Leiningen that link native libraries into projects. But I abandoned that years ago because I stopped doing Clojure development and now I don’t like that declarative approach any more so I am not going to revive it. For my particular need though (LLVM) I did find this, which seems to work out of the box with Clojure. Very nice! https://github.com/bytedeco/javacpp-presets/tree/master/llvm

ghadi15:12:58

I've used JNR successfully, and it claims to be close to JNI, but still high-level.

pesterhazy16:12:00

When using the new clojure cli tool, is there a way to pick a clojure version from the command line?

pesterhazy16:12:25

Using a deps.edn with {:deps {org.clojure/clojure {:mvn/version "1.9.0-RC2"}}} works, but I'm wondering if there's a cli switch

bronsa16:12:12

you can use aliases in deps.edn and then pick which clj version to use by picking a different alias

pesterhazy16:12:41

@bronsa but no way to pick the clojure version from the command line without a deps.edn?

bronsa16:12:56

what do you mean?

bronsa16:12:00

not without

pesterhazy16:12:15

I think that would be a useful feature (boot can do it)

pesterhazy16:12:37

especially given the intended use case of playing around with code from the repl

pesterhazy16:12:57

which version does it pick anyway - is it hardcoded to 1.8.0?

Alex Miller (Clojure team)16:12:59

or I guess, nothing is hardcoded in the tool. The installation files do hardcode the default version.

Alex Miller (Clojure team)16:12:18

there are a series of configuration files that are combined - the default is defined in the installation deps.edn

Alex Miller (Clojure team)16:12:31

if you wanted to, you could define a set of aliases in your config-level deps.edn (usually in ~/.clojure/deps.edn) - then you could use them from anywhere by specifying them at the command line

Alex Miller (Clojure team)16:12:33

{:aliases {
   :1.7 {:override-deps {org.clojure/clojure "1.7.0"}}
   :1.8 {:override-deps {org.clojure/clojure "1.8.0"}}
   :1.9 {:override-deps {org.clojure/clojure "1.9.0-RC2"}}
}}

Alex Miller (Clojure team)16:12:00

then do clj -R:1.7 / clj -R:1.8 / clj -R:1.9

Alex Miller (Clojure team)16:12:24

also btw, there is a #tools-deps room where I watch for questions on this stuff

pesterhazy16:12:56

@alexmiller cool, I'll move it over there then

grzm17:12:02

http://clojurians-log.clojureverse.org seems to have stopped logging about a month (2017-11-16) ago. Is this expected?

eriktjacobsen21:12:09

Wow, that is troubling. Does anyone know who maintains that?

seancorfield19:12:52

@plexus is the person to ask about that I believe?

ghadi19:12:08

@ingesol instrument's argument needs backtick not single quote

ghadi19:12:15

That way you are instrumenting user/myinc not myinc

ghadi19:12:05

`myinc not 'myinc

ingesol19:12:22

oh, didn't know that. But tried now by reloading the whole code in the REPL with backtick, still no instrumentation...

ingesol19:12:34

(st/instrument `myinc)

ingesol19:12:14

I must be doing some silly basic mistake, since I'm the only one having this problem 🙂

ingesol19:12:48

The return value from instrument is [], is that correct?

ingesol20:12:56

I think I figured it out now, the backtick was the solution, I just had some issues referencing vars correctly in repl ns. Thanks so much, @ghadi!

qqq21:12:37

this seems true in practice, but is it guaranteed that the name function, when applied to a qualified keyword, always strips off the namespace, so that (name ::foo/bar) is always equiv to (name :bar) and always equiv to "bar" ?

seancorfield21:12:40

@qqq Given the docstrings for name and keyword, I'd say yes...

seancorfield21:12:09

clojure.core/name
([x])
  Returns the name String of a string, symbol or keyword.
clojure.core/keyword
([name] [ns name])
  Returns a Keyword with the given namespace and name.  Do not use :
  in the keyword strings, it will be added automatically.

seancorfield21:12:37

and

clojure.core/namespace
([x])
  Returns the namespace String of a symbol or keyword, or nil if not present.

seancorfield21:12:58

They are consistent about what they called "name" and what they call "namespace".

bfay22:12:51

If I partition over a sequence, and the 'n' I'm using is larger than the size of the sequence, partition returns an empty list.

(partition 100 (range 10)) => ()
I can see how that's important if you're expecting your groups to be a fixed size, but is there a similar function that returns any remainders?

ghadi22:12:19

see partition-all

ghadi22:12:41

there are other arguments to partition that can help, too.

bfay22:12:11

I wanted the return to be:

((0 1 2 3 4 5 6 7 8 9))

bfay22:12:35

Ah, there's actually a comment from Rich Hickey on this: https://groups.google.com/forum/#!msg/clojure/ZK0zRhAjeJ8/4sM2tDz4_iAJ "The current logic pushes the onus on the producer of the data, where it belongs, rather then the consumer." I'll go with that.

eggsyntax22:12:19

@bfay you can get the behavior you want by providing an empty padding collection. You need to specify a step size to satisfy the arity, so it'd be (partition 100 100 [] (range 10))

tacosundae22:12:55

Is there an idiomatic/performant way to select a random element from a hash-set? (rand-nth (seq set-value)) doesn't feel right.

kgofhedgehogs22:12:07

Hi all. I want to write function to get data with (require '[korma.core :as kc]).

(defn get-message*
  ([query] (kc/select message (kc/where query)))
  ([query & fields] (kc/select message (kc/where query) (apply kc/fields fields))))
It looks ok, but. select is a macro and it want to use it like this:
(kc/select message (kc/where query) (kc/fields field1 field2))
And as result it breaks here: (apply kc/fields fields). How to deal with this?

noisesmith22:12:01

to generate a call to a macro, you usually need to create another macro

kgofhedgehogs22:12:27

Yeah, I thought about macro-apply. Is there one? I dunno how to write it

noisesmith22:12:53

`(kc/select ~message (kc/where ~query) (kc/fields [email protected]))

noisesmith22:12:46

just remember that to do a multi-arity macro you need to make a macro that expands to another call to the macro

noisesmith22:12:51

you can’t just call it the normal way

noisesmith22:12:58

oh, looking again, I probably did message wrong there

kgofhedgehogs22:12:14

Yeah, I noticed

kgofhedgehogs22:12:28

Also. How I should to eval this? Call eval?

noisesmith22:12:51

oh - right - to do this with runtime generated data…

kgofhedgehogs22:12:30

(defn get-message*
  ([query] (kc/select message (kc/where query)))
  ([query & fields] (eval `(kc/select message (kc/where ~query) (kc/fields [email protected])))))

noisesmith22:12:38

ideally you can find a version of this that doesn’t require macros on runtime generated collections, but if that’s the best option all things considered yeah you end up with eval

kgofhedgehogs22:12:20

I also can create raw sql query without macros.. Ill think about this later

tacosundae22:12:13

Is there an idiomatic/performant way to select a random element from a hash-set? (rand-nth (seq set-value)) doesn't seem to perform very well. It is about the same as shuffling a seq and taking the first element.

tacosundae22:12:08

It feels like there should be a much faster way of doing this.

seancorfield22:12:56

@kgofhedgehogs Another option is to stop using Korma (it's no longer maintained, is it?) and use HoneySQL instead with clojure.java.jdbc -- HoneySQL is designed to let you compose query fragments together.

seancorfield22:12:29

@tacosundae a hash set doesn't implement anything with an O(1) access guarantee, just O(n), so I don't think there's anyway to access those elements except sequentially...

tacosundae22:12:34

I see, thanks!

kgofhedgehogs22:12:17

@seancorfield, I like korma for table relationship definitions and as result some features like "with". Is there something like korma what is still maintaining ?

seancorfield22:12:05

@kgofhedgehogs I think that whole entity-relationship thing kinda flies in the face of Clojure's raw data approach... I suspect there are ORM-like alternatives to Korma but I doubt any of them are well-maintained: it's just not really idiomatic (IMO).

kgofhedgehogs22:12:06

hm.. Okay. @seancorfield, honeySQL only library what I should consider to use?

tbaldridge16:12:30

I've been using HugSQL lately, https://www.hugsql.org/

tbaldridge16:12:56

Kindof the best of all worlds, imo. The more you try to make SQL act like Clojure or any other languge, the more pain you're going to have

tbaldridge16:12:11

better to use SQL for what its good for (Querying data) and Clojure for what it's good for.

seancorfield23:12:53

(and then org.clojure/java.jdbc for executing the queries -- which you're almost certainly already using under the hood via Korma)

kgofhedgehogs23:12:22

yeah, I use jdbc in korma. Very old version of jdbc (0.3.7 instead of last 0.7.3)

seancorfield23:12:47

Lots of changes since then -- another reason to avoid Korma!

seancorfield23:12:21

0.7.3 brings reducible queries, for example!

kgofhedgehogs23:12:42

uhh, I already want to switch to HoneySQL

kgofhedgehogs23:12:59

Okay. Another question, It is still related to korma

kgofhedgehogs23:12:07

$ psql testdb
testdb=> \d message
                                         Table "public.message"
     Column      |           Type           | Collation | Nullable |               Default               
-----------------+--------------------------+-----------+----------+-------------------------------------
/* snipped */
 time_sent       | timestamp with time zone |           | not null | 
/* snipped */

$ lein repl
project.scratch=> (require '[korma.core :as kc]
             #_=>          '[clj-time.core :as t]
             #_=>          '[clj-time.coerce :as tc])
nil

project.scratch=> (let [timestamp (tc/to-sql-date (t/now))
             #_=>       msg {:time_sent timestamp, /* snipped */}]
             #_=>   (println "Trying to insert: " (:time_sent msg))
             #_=>   (println "What is inserted: " (:time_sent (kc/insert message (kc/values msg)))))
Trying to insert:  #inst "2017-12-06T22:43:58.074-00:00"
What is inserted:  #inst "2017-12-06T22:00:00.000000000-00:00"
nil

kgofhedgehogs23:12:27

Im using postgresql. Im trying to insert timestamp into table, but I get time without minutes and seconds. Why?

kgofhedgehogs23:12:33

Could it be Korma?

hiredman23:12:50

I am immediately suspicious of / snipped /

kgofhedgehogs23:12:21

Okay, Ill return snipped now

seancorfield23:12:31

I suspect because you're converting it to a SQL Date rather than a timestamp and the JDBC conversion strips the time portion (modulo the TZ adjustment)

hiredman23:12:35

or where the timestamp is being generated and written

kgofhedgehogs23:12:14

@hiredman Timestamp generating in let.

hiredman23:12:31

yeah, but where is it written to the database?

kgofhedgehogs23:12:09

(kc/insert ...

hiredman23:12:40

and what does kv/values return?

kgofhedgehogs23:12:54

Hm, ill chek it now

kgofhedgehogs23:12:56

@hiredman

project.scratch=> (kc/values message {:account_id 3, :dest_account_id 4, :time_sent (tc/to-sql-date (t/now)), :content "hello"})
{:table "message", :name "message", :pk :id, :db nil, :transforms (), :prepares (), :fields [], :rel {"account" #object[clojure.lang.Delay 0x47aa5b5b {:status :pending, :val nil}], "chat" #object[clojure.lang.Delay 0x2b69f1d1 {:status :pending, :val nil}]}, :values [{:account_id 3, :dest_account_id 4, :time_sent #inst "2017-12-06T23:11:24.302-00:00", :content "hello"}]}

kgofhedgehogs23:12:12

It contains not stripped time

kgofhedgehogs23:12:04

@seancorfield, how can I check this?

seancorfield23:12:55

FWIW @kgofhedgehogs We use tc/to-date rather than tc/to-sql-date in our code.

kgofhedgehogs23:12:38

@seancorfield with to-date it throws an error

project.scratch=> (kc/insert message (kc/values {:account_id 3, :dest_account_id 4, :time_sent (tc/to-date (t/now)), :content "hello"}))
PSQLException Can't infer the SQL type to use for an instance of java.util.Date. Use setObject() with an explicit Types value to specify the type to use. org.postgresql.jdbc2.AbstractJdbc2Statement.setObject (AbstractJdbc2Statement.java:1934)

seancorfield23:12:53

(but then we have our app servers and DB servers all set to UTC and our MySQL DB configured to run in UTC as well -- pretty much the only sane setup!)

seancorfield23:12:28

Seriously? The PostgreSQL JDBC driver can't convert from a Java Date to a timestamp with time zone? Wow...

seancorfield23:12:16

Then try tc/to-sql-time which will produce a java.sql.Timestamp instead...

kgofhedgehogs23:12:47

Hurray. this works. btw, it was very dumb to confuse with to-sql-date and to-sql-time

seancorfield23:12:42

Nah, not dumb. Java's various dates and date/times are ... peculiar and annoying.