Fork me on GitHub
#clojure
<
2018-05-04
>
qqq00:05:15

I keep on forgetting -- how do we check if an object is a (float-array) ?

Alex Miller (Clojure team)01:05:51

the class name will be [F

qqq03:05:56

@alexmiller: are you sugesting something like:

(let [fa (float-array 0)
      fac (class fa)]
  (defn float-array? [obj]
    (= fac (class obj))))
?

seancorfield03:05:01

@qqq (Class/forName "[F")

seancorfield03:05:32

so

(defn float-array? [obj]
  (= (class obj) (Class/forName "[F")))

seancorfield03:05:04

That's how you get the class of any Java array...

seancorfield03:05:26

[B is array of byte, [C is array of character, and the [Ljava.lang.String; for arrays of various object types.

seancorfield03:05:58

(the odd one out is [J which is array of long)

igrishaev06:05:41

Morning Clojurians! When checking the incoming JSON data with Spec for REST API, what would be the best way to compose a human-readable message that clearly describes a reason of failure? Do you use spec-tools or any other library, or maybe your own code? For example, if a field :foo is missing, I’d like to compose a message The field 'foo' is missing in parameters. The problem is, inside the data structure that s/explain-data returns, the :in key holds an empty vector and the :pred is a quoted form that looks smth like this: (clojure.core/fn [%] (clojure.core/contains? % :foo)). Well, it wouldn’t be a problem to parse it, but after all I found my own code a bit complex and fragile. Any thoughts?

👍 4
igrishaev07:05:55

Right, already looking through…

dnaeon07:05:00

How would you translate the following code into Clojure?

dnaeon07:05:10

@U1WAUKQ3E thanks, will have a look at it.

qqq08:05:47

(let [a (atom nil)]
  (locking a
    (locking a
    (println "no deadlock"))))
this does not deadlock when I try it locally; is this guaranteed to not deadlock? i.e. it's okay for a thread to acquire a lock we arleady have

leonoel08:05:09

yes, guaranteed by the JVM. locks are reentrant => acquired per-thread, not per-invocation

4
lmergen09:05:59

how exactly do deps.edn git dependencies work ? i'm depending upon a lein-based project that's not yet in clojars. is it required to add a deps.edn to the projects you depend upon as well ?

Alex Miller (Clojure team)11:05:38

Dependent projects are inspected to find a project manifest. It knows how to find and use deps.edn and pom.xml

Alex Miller (Clojure team)11:05:03

It does not know how to use project.clj

Alex Miller (Clojure team)12:05:09

If the project has no deps you can force it to use deps.edn with :deps/manifest :deps in your dep coord and that reader is tolerant of a missing file

borkdude09:05:54

What’s a nice library to make colorized output with Clojure for the console?

borkdude09:05:06

Since GraalVM is in town, I feel like making some cmd line fun

triss10:05:44

when concatenating a list of lists do you prefer reduce concat or mapcat identity?

pesterhazy10:05:21

or even (apply into [] xs), when you want eager evaluation

👍 4
pesterhazy10:05:15

or (apply concat xs) for laziness

lmergen10:05:26

i would prefer reduce concat, the intent is more clear to me than mapcat identity

rauh11:05:11

Or (into [] cat xss), (reduce into [] xss) probably the latter one is fastest

👍 4
lmergen11:05:49

@borkdude jitpack is pretty brilliant! very simple to set up

😃 4
tengstrand12:05:44

I’m running ‘lein test’ from a Clojure program by calling https://clojuredocs.org/clojure.java.shell/sh. Is there a good way of printing the output from the tests while they execute? If the tests takes a long time to execute, then you miss the feedback till they have finished (and the output from sh can be printed out). I found a way of doing it in Java, but it would be nice if I could use clojure.java.shell/sh: http://viralpatel.net/blogs/how-to-execute-command-prompt-command-view-output-java/

Alex Miller (Clojure team)12:05:46

@triss do not use reduce concat as concat builds a lazy stack that can blow your stack later when you first evaluate it. https://stuartsierra.com/2015/04/26/clojure-donts-concat

👍 4
triss12:05:19

thanks @alexmiller! that blog looks brilliant.

danm12:05:29

Heh, I knew there was a reason we used mapcat identity, but I had forgotten what it was 😉

jmckitrick13:05:40

I have a colleague who really likes Scala’s Slick library, which lets you use scala’s map/filter/for operations on a database as if it were a native collection. Is there anything like that for Clojure?

dpsutton13:05:46

on db side or once the collection is back in your code?

dpsutton13:05:23

sounds a bit like linq to sql if its on the db side. but those can make some pretty bad queries.

jmckitrick13:05:34

@dpsutton Yes, it’s like linq. He’s intrigued by Clojure, but loves the ‘linq to sql’ nature of Slick. He just finished rewriting a bunch of queries to make them work efficiently, and said by simply querying with a set of id’s rather than a single id made everything batchable and flat.

jmckitrick13:05:52

So he hasn’t run into any queries that make him doubt Slick yet.

jmckitrick13:05:05

And he’s a sharp guy, so he might figure out how to make it work anyway.

dpsutton13:05:06

ah ok. that sounds good advice in general (set versus n queries i guess which sql can handle just fine). I don't know of anything like that but i would be interested if there were

jmckitrick13:05:34

He’s challenged me to reimplement his slick/scala solution in clojure, so I’m scrambling for ideas, lol.

sveri13:05:34

@jmckitrick what does his slick code look like? Or do you mean slick in general?

jmckitrick13:05:06

So you would do something like this:

jmckitrick13:05:21

val results = users.filter(_.roles.contains(Administrator.toString))

jmckitrick13:05:53

And that operates in a typesafe manner on the users collection.

jmckitrick13:05:27

val query = for {
user <- MyTables.User if user.userId inSet userIds
rolePerms <- MyTables.RolePerms if rolePerms.roleId == user.roleId
perms <- MyTables.Perm if perms.permId == rolePerms.permId
} yield {
( user, perms.name)
}

jmckitrick13:05:35

Sorry for pasting scala here, lol.

jmckitrick13:05:15

So that for comprehension generates a query you can then run and it acts much like a native collection.

jmckitrick13:05:18

So it gives you the flexibility of SQL with the typesafe nature of native collections and their operations, without being as heavy as Hibernate, for example.

jmckitrick13:05:20

@sveri I’m trying to find an equivalent in Clojure that won’t require us to write raw SQL, but isn’t just another DSL on top of SQL.

jmckitrick14:05:32

I might check that out. It’s still very new, and it seems to require a lot of backend wiring to make the magic work. While Slick just lets you generate code from your schema, and you’re off and running.

myguidingstar15:05:11

here's the Walkale version for your Scala code above:

;; schema config, you write it once and can forget it until next time you change your schema
{:columns [:user/id :user/name :permisison/name]
 :idents {:users/all "user"}
 :joins {:user/permissions [:user/role-id :role-permission/role-id
                            :role-permission/permission-id :permission/id]}}

;; your query
;; you see, you can forget how this join is built (which tables and columns involved) and just mention its name
[{(:users/all {:filters [:in :user/id #{1 2 3}]})}
 [:user/id :user/name
  {:user/permissions [:permission/name]}]]

👍 4
tbaldridge13:05:49

@jmckitrick pigpen does this for hadoop

jmckitrick13:05:28

I’m reading now

sveri13:05:33

@jmckitrick Unfortunately I am the wrong one to ask, I was just curious for an example and it reinstates my opinion of ORMs / SQL DSLs: https://www.reddit.com/r/Clojure/comments/8gw75s/orm_and_luminus/dyfj2d2/

tbaldridge13:05:55

But in general you do this differently in Clojure. My recommendation in general though is to take a data-driven approach to this. Don't try to do any work in map or filter. Instead have them compose an AST. That's what linq does anyways. Linq IQueryables are simply ASTs composed by the execution of the program. They are then handed to the linq provider that post processes them into something the datastore can understand

tbaldridge13:05:00

In both Clojure and Linq in C# though you'll hit the same problem, arbitrary predicates will cause problems:

tbaldridge13:05:05

(filter older-than-30 some-db) doesn't work because you have no way to know what older-than-30 does. (filter #(> (:age %) 30) some-db) can work, but requires you to parse the predicate function.

tbaldridge13:05:53

In short you have a big problem here: Clojure doesn't have any sort of staged compiler facilities. Something like Scala's LMS is really hard to duplicate in Clojure without rewriting the entire runtime.

jmckitrick14:05:24

That’s not very encouraging 😞

tbaldridge14:05:03

Scala is really advanced when it comes to this sort of staged compilation. And Lisps approach metaprogramming from a completely different angle.

tbaldridge14:05:03

Without types, or compiler hooks there's little that can be done here aside from writing DSLs. Another approach is given here: https://www.cs.purdue.edu/homes/rompf/papers/amin-popl18.pdf But this method requires a fully polymorphic standard library, or a interpreter for the host language. Sadly we have neither in Clojure.

myguidingstar15:05:11

here's the Walkale version for your Scala code above:

;; schema config, you write it once and can forget it until next time you change your schema
{:columns [:user/id :user/name :permisison/name]
 :idents {:users/all "user"}
 :joins {:user/permissions [:user/role-id :role-permission/role-id
                            :role-permission/permission-id :permission/id]}}

;; your query
;; you see, you can forget how this join is built (which tables and columns involved) and just mention its name
[{(:users/all {:filters [:in :user/id #{1 2 3}]})}
 [:user/id :user/name
  {:user/permissions [:permission/name]}]]

👍 4
eraserhd17:05:22

OK, so... does anybody feel like they're good at organizing things into namespaces for a project? I'm four years into Clojure, and in spite of knowing the common wisdom and pithy sayings, I'm still not liking what I produce.

eraserhd17:05:36

I'm starting to wonder if it's maybe more like having a catalog of patterns to choose from than knowing some simple rules. Yes?

noisesmith17:05:40

Zach Tellman's Elements Of Clojure can provide some insight on the topic

👍 4
eraserhd17:05:20

Is it the last part, on model of abstractions or some such? I'll admit that I haven't finished that bit (love the book a lot tho).

mfikes17:05:58

I would say it is the "names" part, which namespaces affect

dpsutton17:05:35

He's also got potemkin which has a macro you might like. It can import and expose functions as if they were in that namespace. This way you can separate the organization of your code as the developer from the organization exposed to the consumer

noisesmith17:05:54

@eraserhd I had the second half of the book in mind, "indirection" and "composition" - but names come into play as well in retrospect

noisesmith17:05:31

@dpsutton potemkin is sometimes useful but can also make code brittle (vulnerable to breaking if ns load order doesn't happen correctly is what I've seen)

4
noisesmith17:05:44

I'd call it a useful patch-up to make something work if there's a lib you can't control and it can be simpler than forking, I wouldn't call it a good strategy for organizing your own library

eraserhd17:05:20

I've used potemkin before, for hiding all sorts of algorithmic bits and breaking up a namespace. What would a better strategy be in this case?

sveri17:05:02

What I do most of the times is to have packages like db, routes, service, websockets, core, common, components, views. And in the packages I have namespaces grouped by business objects like user, address, authentication, ...

noisesmith17:05:00

you can use naming and ordering of declaration vs. definition within a namespace to clarify without having to hide something in a separate ns

eraserhd17:05:40

@sveri One of the pithy sayings is about, "this is not OO, don't organize things into objects." I'm not sure that's good advice, though.

hiredman17:05:43

I wouldn't put much stock in namespaces, it is useful to slice up your code in to different parts, but beyond the sort of trivial extant, I am not sure it matters how you slice it up

8
bronsa17:05:08

I don't understand the obsession with having just a few functions per namespace

👍 16
noisesmith17:05:11

also there's the pattern of defining a protocol or multimethod as the public interface, and then having a separate implementation namespace that is only for defining the object implementing those abstractions (and no other code that an end user would use directly from that source)

hiredman17:05:15

namespaces are a static phenomenon

bronsa17:05:46

just avoid documenting/exposing internal functions

bronsa17:05:58

put them in a separate namespace if you care to

eraserhd17:05:04

@noisesmith well, I'm actually using Vim fold markers in one of my namespaces, and that makes this strategy tolerable to me (although it is editor-dependent). But Tellman does call out the burden of a large namespace.

noisesmith17:05:06

yeah - in many many cases I've seen attempts to cosmetically groom ns contents lead to more errors and misunderstanding than a bigger namespace with good ordering / naming would have

sveri17:05:32

@eraserhd For me its not about OO vs functional, but how can I find what I am looking for the fastest way and I have been sticking to this structure for many years now, which makes it convenient for me.

👆 4
hiredman17:05:38

obsession with namespaces also neglects the more important organization of your system at runtime

eraserhd17:05:16

@hiredman So, if namespace organization isn't important, but runtime organization is, how do you communicate (say, to new engineers) how the system works and where to find things?

eraserhd17:05:56

Or, also, to your future self.

bronsa17:05:04

@sveri good naming and code organization within the namespace should solve discoverability, having a namespace hierarchy is not the solution imo, you're just creating more bags to look into rather than making it easier to find stuff in the one bag where stuff belongs

hiredman17:05:11

have something that pulls together the puts the parts in to a thing and runs them, have something thing you can point to and say "this is the thing running"

eraserhd17:05:41

Ah, so like a central dispatch or a central middleware stack, or a central bootloader, or what not?

eraserhd17:05:21

OK, I like this.

eraserhd18:05:56

(as awkward as middleware stacks are for data plumbing, I do like them for this reason)

sveri18:05:57

@bronsa Do you have an example for this I can look at? Apart from that I would disagree, because namespace organization for me is part of code organization and good naming. I dont think you can do one thing without the other.

bronsa18:05:29

@sveri clojure.core? :)

bronsa18:05:33

consistent naming and coherent function signatures make having over 600 public functions in a single namespace not a problem

bronsa18:05:07

I'm not suggesting every namespace should be that big but it's a pretty good example (even tho code-org size it suffers a bit due to bootstrapping)

Alex Miller (Clojure team)18:05:40

I think clojure.core is an outlier for any such discussion

👆 4
sveri18:05:44

@bronsa I'd consider clojure.core consistent with my advice 😄 I think one has to differentiate between a library and an application. Libraries usually have few entry points, filled with the accessible functionality while applications have "one" (disregarding REST, etc) entry point and most of the stuff is "internal".

bronsa18:05:26

I was using clojure.core as a counter to the argument that discoverability/ease of use follows from small namespaces

Alex Miller (Clojure team)18:05:38

for me, I start getting that uneasy feeling when my namespace gets bigger than 1000 LOC and a median is probably like 200 LOC.

bronsa18:05:28

yeah, most of my namespaces are complete after at most 1k locs

sveri18:05:41

Btw. I never argued about size of namespaces, but only structure of them.

bronsa18:05:08

ok, I misunderstood then

Alex Miller (Clojure team)18:05:43

I prefer structure to be pretty flat

👍 8
sveri18:05:49

No problem, I have the talent of disexpressing myself 🙂

Alex Miller (Clojure team)18:05:59

I’ve been trying to answer this question about structure well and even after 8 years of full-time Clojure I have hard time articulating it as I don’t usually think about it that much. I suspect that’s probably the result of spending a decade or so in big Java projects before that though

eraserhd18:05:40

Hmm, interesting.

sveri18:05:46

I think the fact that clojure disallows circular dependencies in itself is very helpful already when organizing clojure code. Because of that a path where the data flows will form almost by itself.

bronsa18:05:39

having no implicit forward declares is also one of my favourite things (which I love in ocaml too)

bronsa18:05:05

when people put their api entrypoints at the top and declare all the internal stuff, it makes me super sad

hiredman18:05:08

namespace structure is useful till it isn't, most of the time when I've seen people try to fix issues with code base legibility by moving namespaces around, it doesn't help, because the real issue is runtime organization, which namespaces don't deal with at all

4
sveri18:05:37

@hiredman what exactly do you mean with runtime organization?

eraserhd18:05:42

@alexmiller Hmm, I have a bunch of Java and Way Too Much C++, and weirdly, organizing C++ is like an artform (which makes it more like Clojure, I guess), but organizing Java feels very much "on rails" to me.

bronsa18:05:56

that's because java mostly has a 1:1 relation between classes and files

hiredman18:05:00

@sveri when your system exists and is running, are all the parts of it gathered together? do you have a clear means of starting and stopping your system? or do you just have your -main call some functions in different namespaces and stuff starts happening?

hiredman18:05:12

are the interfaces to different parts of your system well defined enough that you can swap them out?

sveri18:05:59

I see, I think that is a different problem though. But yea, I agree, moving namespaces around usually is a sign of despair. A kind of "let's throw dirt onto the wall and hope some of it sticks" solution.

hiredman18:05:20

I see it more as a "if you give someone a hammer, they will try and fix everything using nails"

hiredman18:05:55

so namespaces are great, but keep it flat, keep it a tree, and don't sweat the names beyond some reasonable lower bound. maybe start tagging your namespaces or something

hiredman18:05:05

one issue with namespaces is you get a single name for things, and even though technically clojure doesn't treat them as hierarchical, people almost always treat them that way for organizing

hiredman18:05:12

and the issue with hierarchies like that is it is not uncommon for people to think slightly differently about things, so they have different hierarchies in mind

hiredman18:05:21

in contrast tagging is usually a flat thing where something can have different sets of tags and different people can tag it differently, how you would use that to make a coherent namespace system, I have no idea

eraserhd18:05:18

ah, interesting.

eraserhd18:05:43

Well, I do like systems which have only one global namespace, but it kind of precludes using libraries.

rymndhng19:05:41

how is metadata carried around with transformations to persistent data structures? is appears with some testing of assoc and merge it's always the meta of the first arg that carries forward

mikerod20:05:48

@U09U89Y6Q I don’t know of any good and definitive sources on this topic.

mikerod20:05:45

I’d say that metadata preservation typically works “as you may expect” with may of the clojure.core functions. Most of the times when there are multiple objects with metadata involved and are going to be merged in some way to create a single structuer, the first from left-to-right is the metadata preserved

mikerod20:05:04

I would not expect any built in features to automatically try to merge or combine metadata between data structures for you though

mikerod20:05:40

Also, clojure.walk has had some issues in the past (and I think still a few) where metadata can be lost. There are various edge cases. Anytime you have particular worry about losing metadata, I think you should be a bit careful and test things out.

mikerod20:05:08

If you want some more details on various things, you may try “preserve meta” search in Clojure Jira https://dev.clojure.org/jira/secure/IssueNavigator.jspa?

rymndhng21:05:11

thanks for the links -- yeah it seems like a nice to have, but not a great way to have observability in code

mikerod22:05:11

Yeah, metadata can be useful

mikerod22:05:41

However, I think it should be something carefully considered sometimes - sometimes it makes sense for the data to be “real” data and not meta

triss20:05:24

ok how do I read a .edn file in?

triss20:05:28

As in create a PushbackReader?

triss20:05:17

I wish there was a clojure.edn/slurp

sveri20:05:01

@triss (read-string (slurp "data.clj")) if its not to large.

sveri20:05:50

hups, that should be data.edn.

👍 4
triss20:05:09

got it going. Thanks man!

Alex Miller (Clojure team)20:05:51

make sure you use clojure.edn/read-string

Alex Miller (Clojure team)20:05:05

slurp has the downside of slurping the whole thing into memory first. for a bit more robust impl, see something like https://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/util/io.clj#L22

👍 4
triss21:05:22

thanks all… next question. Has anyone implemented anything like these ‘adverbs’ before in Clojure? I really love the, they can all be emulated easily enough but I wonder if they exist in a library somewhere… I want to be able to do this stuff with symbols/objects etc as well as numbers: http://doc.sccode.org/Reference/Adverbs.html

triss21:05:52

obviously .s is equivalent to map

Alex Miller (Clojure team)21:05:48

f is map-indexed, t I don’t think has a particular function but could be done with a combination, x is for

Alex Miller (Clojure team)21:05:17

t could be done with for or nested map

Alex Miller (Clojure team)22:05:52

tbh I think Clojure has a much better set of abstractions here