I'm curious if anyone has any ideas about the best way to organize protocol declarations vs. implementations. Currently I'm sharing some defrecords and defprotocols between front/back end in a cljc file (ns projectname.protocols.core) and defining the differing implementations in clj and cljs files (ns projectname.protocols.server) and (ns projectname.protocols.front-end). This works, but I was wondering if there is a possibly a better way to organize these

why would the frontend and backend share protocols?

Front-end creates new records/modifies existing records to be sent to back-end to store in DB. Seemed cleaner/more declarative to share protocols. I COULD just use maps, but since the maps are always the same fields, records w/ protocols seemed better. This is the first time I've tried using protocols though, so I'm not sure that was the best way to go.

the thing that doesn't make sense to me is that protocols are behaviors, and you can't really share a behavior in frontend and backend

I guess kind of ... it just seems like something that's very likely to have weird exceptions to the rule

at the moment the only actual protocol methods I'm using on the front-end are ones associated with checking validity. These records are all generated on the server by reading from PDFs, and it's an error-prone process, to say the least, especially when users modify them on the front end in ways that may break them.

It just seemed simpler to user protocols so the methods have the same name and I have less to remember. If there is a better way though, I'd love to know.

if you have to do the same thing on both ends you can use cljc - and then sharing a protocol makes sense

but in my experience it's rare I want the same logic to exist on both sides

Im writing a toy project in clojure that constantly reads messages from a website and checks for the inclusion of a small set of keywords.

I’ve done a pass at it and it works great. Now I want to explore concurrency.

A set of keywords is small e.g. ["lisp" "racket" "scheme" "clojure"] if the message has any of those keywords it should be tagged somehow.

But now I want to have a big list of keyword sets. E.g. [#{lisp racket scheme clojure} #{linux microsoft apple} #{functional imperative logic}…]

How could I leverage concurrency to check each message against each set of keywords?

I understand that future takes a body and runs it in another thread.

Are these threads lightweight?

Would it be problematic to say, launch 100 futures at the same time?

Or are they similar to goroutines/erlang beam threads

I need an ordered container with two ops: 1. concat another ordered container 2. pop off the front element is vector, list, or something else best for this?

probably PersistentQueue + into for concat

though it depends how long the things you're concating are, because that'll be O(1) for the rhs

Does persistent queue add a bunch of expensive synchronization primitives, or is it literally just a queue ?

this is for implementing topological sort, I need to conccat 1. stuff to visit next WITH 2. stuff already on to-visit list

literally just a queue. it's immutable so no synchronisation required

nb, the function clojure.core/concat never returns a PersistentQueue, it can only return a LazySeq, no matter what the input is

to use PersistentQueue you need to use peek, pop, conj, into (and other things builton those). I dealt with a bug where a FIFO (PersistentQueue) was turning into a LIFO (LazySeq) because someone was using rest on it.

At what point are resources loaded? I have a static resource file and I want to read it once. If I read it on integrant system init it works, but if I read it in a def it throws.

(def schema (slurp (io/resource "schema.json")))
;; Throws "Cannot open <nil> as a Reader."

;; while this works:
(defmethod ig/init-key :validator
  [_ _]
  (v/validator (slurp (io/resource "schema.json"))))

I'm looking to do some graph exploration and visualization of some pretty shoddily gathered data, and Clojure seems like a good fit with the interactive programming experience in the REPL. Does anyone know of a good graph visualization library for clojure that could be presented either as html or images?

I'm currently running thru all the top google hits for "graph visualization clojure", but any real experience would be super helplful too! 😃

@emilaasa Maybe this? https://github.com/jasongilman/proto-repl-charts

That looks cool!

The longterm super-solution would be to visualize parts of this data as an interactive web site for everyone that is interested

This week I probably need to be able to just export some images that can be used in a presentation

@emilaasa https://github.com/ztellman/rhizome

however, I consider visualization problem far from solved. to get any level of interactivity, you need to dig through tons of obscure and fragile js, or settle down with whatever graphviz offers, which is good enough for many one-off cases (exactly like your "just export some images that can be used in a presentation")

but as soon as you try anything slightly more ambitious – you are bound to do lots of unpleasant integration, and/or re-implementation

Yeah, think presentation to the VPs of Sales, CEO etc. It needs to look sort of good and convey the right kind of information

Yeah, I've worked with d3js before for example

d3 is awful for this, despite mad respect and recommendations, sadly

Well it was implemented before virtual-dom was a standalone thing so you end up writing code on many levels of abstraction

(again, it is good enough for basic enough use cases)

I totally agree and after googling for a while it feels pretty unsolved. Which I find surprising to be honest since we deal with so much graph data as developers

I appreciate the recommendations @misha and @yonatanel

Wonder if someone tried making a declarative wrapper around d3 with any degree of success

Hi! Quick question on either Clojure Spec or Plumatic Schema: how would you take in a (nested) map of which some fields are UUIDs and turn it into a nested map where those UUIDs are resolved to DB ids (by doing a query)?

Is that something acceptable to do in a conformer? It seems a bit odd to me

@emilaasa I heard some opinions, that it does not make much sense to wrap it, it should be reimplemented instead (do not remember particular arguments behind it though)

@hmaurer sounds odd to do IO in a conformer. I'd do this separately

@noisesmith thank you for this https://gist.github.com/noisesmith/3490f2d3ed98e294e033b002bc2de178

Hi all

what’s the clojure idiom for try/let/catch/finally? I want to finally close a resource if it was created on a let block

(try (let [driver (create-driver...)]
  (catch Exception e)
  (finally (if (driver ) (close driver))))) ;;;wont work because driver is out of scope

You can try with-open https://clojuredocs.org/clojure.core/with-open

@govind that’s for something that has .close right?

Yes. Sorry, I read (close driver) as (.close driver)

Hi all I wonder if I'm reading the doc for clojure.core.async/pipeline-async correctly, specifically the sentence: af must close! the channel before returning. This seems odd to me since my assumption would be that an operation should be launched by af putting the result on the channel, and then closing. Thus closing the channel may happen outside of af's lifecycle. Have I got things wrong?

the expectation is that some other thread is asynchronously putting the result on the channel and closing it

wow, sweet, same ns, separate file:

(in-ns 'clojure.pprint)

(defn print-table

| :common-path |
|--------------|
|   [:a :b :e] |
|      [:a :b] |

@alexmiller that was my understanding as well. shouldn't the wording not imply that af itself should close?

speaking of wording, the last sentence could be simpler

there’s actually a ticket for this https://dev.clojure.org/jira/browse/ASYNC-144

Ah sorry, hadn't checked for it

thank you

@alexmiller the suggested wording is less confusing, indeed

Is there a function that given a subvec will tell me its location in the entire vector?

Nm, realized I could just call (.start ...) and (.end ...) as well (.v ...) to get the original vector. I guess I'll add this to Clojure Docs.

Is gorilla-repl essentially unmaintained at this point? There are close on a dozen PRs that are several months old and the libraries it uses are quite out of date. Notably Vega, the visualisation grammar. Slightly sad considering that it is an otherwise a great tool.

https://gist.github.com/nathanmarz/9d8805d5e7d9c689cd629bd0551e84f1#gistcomment-2187257 @alexmiller where can I read in depth about "why vector is not sequence", etc.? many of these differences at the first glance look like implementation detail, and not very obvious, however those are very important.

@misha It's mostly about the performance guarantees for concrete collection types.

This is probably the best place to start @misha https://clojure.org/reference/data_structures -- it talks about the interfaces and performance guarantees.

Question: is there an existing utlity function for clojure spec and/or spec-tools to walk a spec and data? (e.g. for each subpart of a piece of data apply a function to the spec + subpart)

like clojure.walk but traversing the spec as well

Are there any build tools that 1) support clojure well 2) support make-style only rebuild things that changed?

@arohner Juxt are working on this, which sounds like what you’re looking for: https://github.com/juxt/mach

interesting, thanks!

Are there options for compiling clojure programs to native executables/bin rather than jars?

not good ones - if you do that you force the user to download the full jvm as part of your app - which is possible but also wasteful considering they are likely to need one already for at least one other thing they use

unless an uberjar counts and the user supplies their own jvm? that can work

I was thinking more along the lins of a clojure compiler down to C, I heard something about a native clojure compiler but can't find it. I think it was for rust.

there are clojure-like languages that do that

ferret is the one that I've seen the most about recently

http://ferret-lang.org/

ooh

thanks

obviously by restricting yourself to not using the JVM, you end up writing very different programs (binding to C libraries or gpio/serial stuff)

@theeternalpulse many people have wanted to / tried such a thing, and most of the projects fizzled. It seems like ferret might be more usable than the rest though. But idiomatic clojure needs a really good gc, which most things that aren’t the jvm don’t have.

IIRC, ferret compiles to C++ and assumes knowledge of C++ as well as Clojure

even the jvm gc is barely good enough for clojure imho

having built some toy programs with ferret, it's a much difference experience than the clojure apps that I typically write. That said, if what you need is to deploy on a raspberry pi or some microcontroller, it's not a bad environment

I previously used chicken scheme and pixie-lang for some of the random toys I've rewritten in ferret

yeah - that’s cool, I wouldn’t go in expecting that to be clojure per se, but it sounds a lot nicer than using C++ by itself (but hey I hear newer C++ can be decent…)

unless you need very low memory requirements, I'd probably steer towards clojurescript on node before ferret

it'll work well enough on a raspberry pi, although probably not run too well on anything smaller

and it has more momentum than ferret afaict

Ferret is mostly for embedded programming.. it doesn't have persistent data structures and a whole ton of other critical features. It's basically a way of writing C++ with clojure syntax, rather than the language itself.

Didn't realize ferret's data structures weren't persistent. That's good to know. I assume they're also not immutable then?

ah, yeah. I just was looking for something more accessable for a simple cli program, and didn't know if the jvm solution was best for general distribution to most platforms

not having immutable data is definitely one way to solve the “clojure needs the best gc ever” problem

@noisesmith transients solve that problem a bit, as could reference counting, but I don't think anyone has tried that yet. If inside assoc you test to see if the refcount = 1 then you could safely mutate the structure instead of performing a persistent update.

but refcounting becomes very expensive with multicore systems.

interesting… it seems inherent in the usage of persistent data structures in small functions that are meant to compose that a gc would get stressed, but maybe I lack the imagination

the problem is reference count tracking is not cheap, so if you want to optimize it you delay and batch it which of course then turns in to garbage collection

@hiredman pretty much

there is that whole paper

even the highest end "lock free" hashmaps in C++ take 1-2 CAS per read/write

https://www.cs.virginia.edu/~cs415/reading/bacon-garbage.pdf

also: https://github.com/tpn/pdfs/blob/master/Down%20For%20The%20Count%20-%20Getting%20Reference%20Counting%20Back%20in%20the%20Ring%20(rc-ismm-2012).pdf

So anyway, that's what made me say "meh" about Ferret. There's not a whole not of information on how the authors plan to solve these problems.

for that matter, saying "we're gonna use a GIL" is an acceptable answer in my book, but I'd just like to know before someone says "we're Clojure on C++!"

if there’s a GIL I’d skip the bs and just use OCaml, heh

in general, when you hear "look, its clojure on some other runtime" treat like "look here's a lisp on some other runtime"

the response is "oh, what a cute toy, isn't lisp just great"

if I recall, ferret isn't even the first clojure to c++ compiler I've seen

huh, ferret may actually be the continuation of the blog posts I half remember seeing about translating clojure to c++ way back when

https://nakkaya.com/2011/06/29/ferret-an-experimental-clojure-compiler/ from back in 2011

what am I doing wrong here?

=> (into [:a] distinct [:b :c :b :c :d])

ClassCastException clojure.lang.LazySeq cannot be cast to clojure.lang.IFn  clojure.core/transduce (core.clj:6599)

I must be missing something simple

(distinct)

oh, right, because I need to init a stateful transducer

well, doesn't matter that it's stateful or not

but you need the transducer, which you get by invoking the 0-arity of distinct

my thought was that if it were not stateful, I could use it directly, and since it needs to be stateful, it needs to be something you can generate, which means calling something

of course, distinct is also a function, so…