Fork me on GitHub
#clojure
<
2017-08-28
>
tjscollins00:08:19

I'm curious if anyone has any ideas about the best way to organize protocol declarations vs. implementations. Currently I'm sharing some defrecords and defprotocols between front/back end in a cljc file (ns projectname.protocols.core) and defining the differing implementations in clj and cljs files (ns projectname.protocols.server) and (ns projectname.protocols.front-end). This works, but I was wondering if there is a possibly a better way to organize these

noisesmith00:08:05

why would the frontend and backend share protocols?

tjscollins00:08:16

Front-end creates new records/modifies existing records to be sent to back-end to store in DB. Seemed cleaner/more declarative to share protocols. I COULD just use maps, but since the maps are always the same fields, records w/ protocols seemed better. This is the first time I've tried using protocols though, so I'm not sure that was the best way to go.

noisesmith00:08:20

the thing that doesn't make sense to me is that protocols are behaviors, and you can't really share a behavior in frontend and backend

noisesmith00:08:41

I guess kind of ... it just seems like something that's very likely to have weird exceptions to the rule

tjscollins00:08:37

at the moment the only actual protocol methods I'm using on the front-end are ones associated with checking validity. These records are all generated on the server by reading from PDFs, and it's an error-prone process, to say the least, especially when users modify them on the front end in ways that may break them.

tjscollins00:08:36

It just seemed simpler to user protocols so the methods have the same name and I have less to remember. If there is a better way though, I'd love to know.

noisesmith00:08:49

if you have to do the same thing on both ends you can use cljc - and then sharing a protocol makes sense

noisesmith00:08:07

but in my experience it's rare I want the same logic to exist on both sides

juanjo.lenero01:08:24

Im writing a toy project in clojure that constantly reads messages from a website and checks for the inclusion of a small set of keywords.

juanjo.lenero01:08:50

I’ve done a pass at it and it works great. Now I want to explore concurrency.

juanjo.lenero01:08:38

A set of keywords is small e.g. ["lisp" "racket" "scheme" "clojure"] if the message has any of those keywords it should be tagged somehow.

juanjo.lenero01:08:04

But now I want to have a big list of keyword sets. E.g. [#{lisp racket scheme clojure} #{linux microsoft apple} #{functional imperative logic}…]

juanjo.lenero01:08:38

How could I leverage concurrency to check each message against each set of keywords?

juanjo.lenero01:08:10

I understand that future takes a body and runs it in another thread.

juanjo.lenero01:08:21

Are these threads lightweight?

juanjo.lenero01:08:34

Would it be problematic to say, launch 100 futures at the same time?

juanjo.lenero01:08:49

Or are they similar to goroutines/erlang beam threads

qqq01:08:14

I need an ordered container with two ops: 1. concat another ordered container 2. pop off the front element is vector, list, or something else best for this?

bfabry01:08:10

probably PersistentQueue + into for concat

bfabry01:08:37

though it depends how long the things you're concating are, because that'll be O(1) for the rhs

qqq01:08:48

Does persistent queue add a bunch of expensive synchronization primitives, or is it literally just a queue ?

qqq01:08:09

this is for implementing topological sort, I need to conccat 1. stuff to visit next WITH 2. stuff already on to-visit list

bfabry01:08:15

literally just a queue. it's immutable so no synchronisation required

noisesmith01:08:10

nb, the function clojure.core/concat never returns a PersistentQueue, it can only return a LazySeq, no matter what the input is

noisesmith01:08:24

to use PersistentQueue you need to use peek, pop, conj, into (and other things builton those). I dealt with a bug where a FIFO (PersistentQueue) was turning into a LIFO (LazySeq) because someone was using rest on it.

yonatanel07:08:32

At what point are resources loaded? I have a static resource file and I want to read it once. If I read it on integrant system init it works, but if I read it in a def it throws.

(def schema (slurp (io/resource "schema.json")))
;; Throws "Cannot open <nil> as a Reader."

;; while this works:
(defmethod ig/init-key :validator
  [_ _]
  (v/validator (slurp (io/resource "schema.json"))))

emilaasa10:08:39

I'm looking to do some graph exploration and visualization of some pretty shoddily gathered data, and Clojure seems like a good fit with the interactive programming experience in the REPL. Does anyone know of a good graph visualization library for clojure that could be presented either as html or images?

emilaasa10:08:06

I'm currently running thru all the top google hits for "graph visualization clojure", but any real experience would be super helplful too! 😃

emilaasa10:08:14

That looks cool!

emilaasa10:08:32

The longterm super-solution would be to visualize parts of this data as an interactive web site for everyone that is interested

emilaasa10:08:42

This week I probably need to be able to just export some images that can be used in a presentation

misha10:08:42

however, I consider visualization problem far from solved. to get any level of interactivity, you need to dig through tons of obscure and fragile js, or settle down with whatever graphviz offers, which is good enough for many one-off cases (exactly like your "just export some images that can be used in a presentation")

misha10:08:42

but as soon as you try anything slightly more ambitious – you are bound to do lots of unpleasant integration, and/or re-implementation

emilaasa10:08:46

Yeah, think presentation to the VPs of Sales, CEO etc. It needs to look sort of good and convey the right kind of information

emilaasa10:08:53

Yeah, I've worked with d3js before for example

misha10:08:25

d3 is awful for this, despite mad respect and recommendations, sadly

emilaasa10:08:03

Well it was implemented before virtual-dom was a standalone thing so you end up writing code on many levels of abstraction

misha10:08:05

(again, it is good enough for basic enough use cases)

emilaasa10:08:58

I totally agree and after googling for a while it feels pretty unsolved. Which I find surprising to be honest since we deal with so much graph data as developers

emilaasa10:08:26

I appreciate the recommendations @misha and @yonatanel

emilaasa10:08:51

Wonder if someone tried making a declarative wrapper around d3 with any degree of success

hmaurer10:08:43

Hi! Quick question on either Clojure Spec or Plumatic Schema: how would you take in a (nested) map of which some fields are UUIDs and turn it into a nested map where those UUIDs are resolved to DB ids (by doing a query)?

hmaurer10:08:56

Is that something acceptable to do in a conformer? It seems a bit odd to me

misha10:08:08

@emilaasa I heard some opinions, that it does not make much sense to wrap it, it should be reimplemented instead (do not remember particular arguments behind it though)

mpenet11:08:32

@hmaurer sounds odd to do IO in a conformer. I'd do this separately

mping12:08:32

what’s the clojure idiom for try/let/catch/finally? I want to finally close a resource if it was created on a let block

mping12:08:48

(try (let [driver (create-driver...)]
  (catch Exception e)
  (finally (if (driver ) (close driver))))) ;;;wont work because driver is out of scope

mping12:08:24

@govind that’s for something that has .close right?

govind12:08:25

Yes. Sorry, I read (close driver) as (.close driver)

pyr12:08:20

Hi all I wonder if I'm reading the doc for clojure.core.async/pipeline-async correctly, specifically the sentence: af must close! the channel before returning. This seems odd to me since my assumption would be that an operation should be launched by af putting the result on the channel, and then closing. Thus closing the channel may happen outside of af's lifecycle. Have I got things wrong?

Alex Miller (Clojure team)13:08:30

the expectation is that some other thread is asynchronously putting the result on the channel and closing it

misha13:08:56

wow, sweet, same ns, separate file:

(in-ns 'clojure.pprint)

(defn print-table
is there a way to specify column content alignment? it aligns to the right atm, want to change it to the left, it is weird to compare seqs this way:
| :common-path |
|--------------|
|   [:a :b :e] |
|      [:a :b] |

misha13:08:01

https://github.com/clojure/clojure/blob/master/src/clj/clojure/pprint/print_table.clj#L23

--- fmts (map #(str "%" % "s") widths)
+++ fmts (map #(str "%-" % "s") widths)
| :common-path |
|--------------|
| [:a :b :e]   |
| [:a :b]      |
opieop

sophiago13:08:45

If you care about performance you could also just use "\t" to insert tabs

misha15:08:58

I don't, given it's dev and repl only use case

pyr13:08:22

@alexmiller that was my understanding as well. shouldn't the wording not imply that af itself should close?

pyr13:08:51

speaking of wording, the last sentence could be simpler

pyr13:08:44

Ah sorry, hadn't checked for it

pyr13:08:46

thank you

pyr13:08:32

@alexmiller the suggested wording is less confusing, indeed

sophiago13:08:24

Is there a function that given a subvec will tell me its location in the entire vector?

sophiago14:08:13

Nm, realized I could just call (.start ...) and (.end ...) as well (.v ...) to get the original vector. I guess I'll add this to Clojure Docs.

jonathanj16:08:18

Is gorilla-repl essentially unmaintained at this point? There are close on a dozen PRs that are several months old and the libraries it uses are quite out of date. Notably Vega, the visualisation grammar. Slightly sad considering that it is an otherwise a great tool.

misha18:08:17

https://gist.github.com/nathanmarz/9d8805d5e7d9c689cd629bd0551e84f1#gistcomment-2187257 @alexmiller where can I read in depth about "why vector is not sequence", etc.? many of these differences at the first glance look like implementation detail, and not very obvious, however those are very important.

seancorfield18:08:39

@misha It's mostly about the performance guarantees for concrete collection types.

seancorfield18:08:54

This is probably the best place to start @misha https://clojure.org/reference/data_structures -- it talks about the interfaces and performance guarantees.

hmaurer18:08:40

Question: is there an existing utlity function for clojure spec and/or spec-tools to walk a spec and data? (e.g. for each subpart of a piece of data apply a function to the spec + subpart)

hmaurer18:08:06

like clojure.walk but traversing the spec as well

arohner19:08:54

Are there any build tools that 1) support clojure well 2) support make-style only rebuild things that changed?

chrisjd19:08:19

@arohner Juxt are working on this, which sounds like what you’re looking for: https://github.com/juxt/mach

arohner19:08:31

interesting, thanks!

theeternalpulse20:08:13

Are there options for compiling clojure programs to native executables/bin rather than jars?

noisesmith20:08:55

not good ones - if you do that you force the user to download the full jvm as part of your app - which is possible but also wasteful considering they are likely to need one already for at least one other thing they use

noisesmith20:08:23

unless an uberjar counts and the user supplies their own jvm? that can work

theeternalpulse20:08:13

I was thinking more along the lins of a clojure compiler down to C, I heard something about a native clojure compiler but can't find it. I think it was for rust.

bja20:08:25

there are clojure-like languages that do that

bja20:08:35

ferret is the one that I've seen the most about recently

bja20:08:56

obviously by restricting yourself to not using the JVM, you end up writing very different programs (binding to C libraries or gpio/serial stuff)

noisesmith20:08:17

@theeternalpulse many people have wanted to / tried such a thing, and most of the projects fizzled. It seems like ferret might be more usable than the rest though. But idiomatic clojure needs a really good gc, which most things that aren’t the jvm don’t have.

bja20:08:27

IIRC, ferret compiles to C++ and assumes knowledge of C++ as well as Clojure

noisesmith20:08:10

even the jvm gc is barely good enough for clojure imho

bja20:08:52

having built some toy programs with ferret, it's a much difference experience than the clojure apps that I typically write. That said, if what you need is to deploy on a raspberry pi or some microcontroller, it's not a bad environment

bja20:08:45

I previously used chicken scheme and pixie-lang for some of the random toys I've rewritten in ferret

noisesmith20:08:57

yeah - that’s cool, I wouldn’t go in expecting that to be clojure per se, but it sounds a lot nicer than using C++ by itself (but hey I hear newer C++ can be decent…)

bja20:08:16

unless you need very low memory requirements, I'd probably steer towards clojurescript on node before ferret

bja20:08:35

it'll work well enough on a raspberry pi, although probably not run too well on anything smaller

bja20:08:51

and it has more momentum than ferret afaict

eriktjacobsen20:08:22

Ferret is mostly for embedded programming.. it doesn't have persistent data structures and a whole ton of other critical features. It's basically a way of writing C++ with clojure syntax, rather than the language itself.

jeff.terrell20:08:36

Didn't realize ferret's data structures weren't persistent. That's good to know. I assume they're also not immutable then?

theeternalpulse20:08:56

ah, yeah. I just was looking for something more accessable for a simple cli program, and didn't know if the jvm solution was best for general distribution to most platforms

noisesmith20:08:11

not having immutable data is definitely one way to solve the “clojure needs the best gc ever” problem

tbaldridge21:08:45

@noisesmith transients solve that problem a bit, as could reference counting, but I don't think anyone has tried that yet. If inside assoc you test to see if the refcount = 1 then you could safely mutate the structure instead of performing a persistent update.

tbaldridge21:08:57

but refcounting becomes very expensive with multicore systems.

noisesmith21:08:01

interesting… it seems inherent in the usage of persistent data structures in small functions that are meant to compose that a gc would get stressed, but maybe I lack the imagination

hiredman21:08:28

the problem is reference count tracking is not cheap, so if you want to optimize it you delay and batch it which of course then turns in to garbage collection

hiredman21:08:07

there is that whole paper

tbaldridge21:08:18

even the highest end "lock free" hashmaps in C++ take 1-2 CAS per read/write

tbaldridge21:08:50

So anyway, that's what made me say "meh" about Ferret. There's not a whole not of information on how the authors plan to solve these problems.

tbaldridge21:08:18

for that matter, saying "we're gonna use a GIL" is an acceptable answer in my book, but I'd just like to know before someone says "we're Clojure on C++!"

noisesmith21:08:52

if there’s a GIL I’d skip the bs and just use OCaml, heh

hiredman21:08:39

in general, when you hear "look, its clojure on some other runtime" treat like "look here's a lisp on some other runtime"

hiredman21:08:58

the response is "oh, what a cute toy, isn't lisp just great"

hiredman22:08:19

if I recall, ferret isn't even the first clojure to c++ compiler I've seen

hiredman22:08:30

huh, ferret may actually be the continuation of the blog posts I half remember seeing about translating clojure to c++ way back when

noisesmith23:08:39

what am I doing wrong here?

=> (into [:a] distinct [:b :c :b :c :d])

ClassCastException clojure.lang.LazySeq cannot be cast to clojure.lang.IFn  clojure.core/transduce (core.clj:6599)

noisesmith23:08:44

I must be missing something simple

bronsa23:08:35

(distinct)

noisesmith23:08:54

oh, right, because I need to init a stateful transducer

bronsa23:08:33

well, doesn't matter that it's stateful or not

bronsa23:08:02

but you need the transducer, which you get by invoking the 0-arity of distinct

noisesmith23:08:24

my thought was that if it were not stateful, I could use it directly, and since it needs to be stateful, it needs to be something you can generate, which means calling something

noisesmith23:08:32

of course, distinct is also a function, so…