Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2021-05-28

Channels

#clojure
<
2021-05-28
>
Ethan Plante02:05:34

Leiningen question: how does the main function work, once you run the app? Does it start at main and go through and evaluate all the code in that file, or does it only run the code you call from within the main function?

seancorfield04:05:20

I answered that on Zulip -- hope that helped? (and this sounds like a good question for the #beginners channel here on Slack).

Karol Wójcik09:05:15

I think I saw last time Clojure XML parser/generator which claims that it's faster than data.xml. Does anybody remember the name of it?

Karol Wójcik09:05:54

Exactly. Thanks!

marrs10:05:28

Can someone tell me what's going on with this problem I'm having? 

(for [x [1 2 3]]
  (println "X" x))

; yields
; X 1
; X 2
; X 3
; (nil nil nil)

(do (println "DO")
    (for [x [1 2 3]]
      (println "X" x))
    (println "DONE"))

; yields
; DO
; DONE
; nil
Any ideas why?

delaguardo10:05:23

for is lazy. You need something to realize it for side effects

thheller10:05:52

use doseq instead of for if you want it just for side-effects

☝️ 3
marrs10:05:15

ah, side-effects!

borkdude11:05:52

@alexmiller @hiredman Re: Clojure AOT problems with GraalVM native-image, A similar problem was reported with depstar and :aot true which has similar behavior to lein's :aot :all. https://github.com/oracle/graal/issues/1613#issuecomment-850325231

Rob Haisfield12:05:40

If we assume a structural editor for Clojure is desirable, what potential barriers would the design need to account for? What awesome things could the design account for?

👍 6
Stuart12:05:04

Do you mean like paredit / parinfer? But more specific to Clojure?

Rob Haisfield13:05:44

Paredit or Parinfer are sources of inspiration! I’m referring to a sort of graphical/textual editor for manipulating and navigating code that prevents syntactic errors.

Rob Haisfield13:05:33

Something that would make it more approachable. The things I’m thinking about may apply to Lisp generally, but I’d prefer to start my explorations with Clojure

p-himik13:05:58

Suppose you need to add a map literal somewhere in the code, like {:a 1}. When you're typing it in a text editor - no matter how advanced the editor is, it's still text, so you will end up having an intermediate step of {:a|} where | is your text cursor (the } was inserted by the editor itself - it's a very common functionality). Something like clj-kondo will likely highlight that map literal as being incorrect. But how an editor can prevent such an error instead of just highlighting it?

Rob Haisfield13:05:51

Ah so are you basically saying that it would need to account for intermediate steps on the way to correct code?

blak3mill3r13:05:52

I've often thought it might be fun to have a way of editing lisp with a VR helmet and something akin to an xbox kinect field sensor

blak3mill3r13:05:24

Waving my arms about to do what lispy/paredit call slurping and barfing, etc

p-himik13:05:32

@U02108ERRU5 I simply don't see any other way around it right now. So the editor will only let you save the file when there are no syntax errors (at least, that's how I understand your initial idea). So between | and {:a 1|} there will be no saves, no ability to evaluate code. But... is it really helpful to justify a new editor by itself? What if you need to enter a big nested data structure, like {:a {:b {:c ...}}} , but you also want to push your work into your Git branch somewhere in the process of entering it to save the progress?

p-himik13:05:10

Don't get me wrong - I'm all up for new tools. I'm just really unsure if an editor with more restrictions will be helpful. IMO it would be much more useful to have handy tools that help you make sense of the current state of your code, your development environment, and the relevant running processes. For example, a functionality to compare the code that has been evaluated most recently with the current code. And maybe it exists somewhere, but I haven't seen it.

Rob Haisfield13:05:50

Hmm only saving when there are no syntax errors is an option. Alternatively, one where it makes it visually clear what the gaps are and what can go in them.

Stuart13:05:18

> For example, a functionality to compare the code that has been evaluated most recently with the current code. Yes! This would be great.

p-himik13:05:22

> one where it makes it visually clear what the gaps are and what can go in them But isn't that exactly what linters like clj-kondo do?

p-himik13:05:44

@U013YN3T4DA Feel free to steal the idea. ;)

Rob Haisfield13:05:17

> For example, a functionality to compare the code that has been evaluated most recently with the current code. Yes! Would show me if function definitions and values are out of date

Rob Haisfield13:05:56

Clj-kondo means there’s something to build on! I just have a general open question about how to visually communicate this stuff as well as possible and I think standard text editors are gonna be limited there

p-himik13:05:48

Also worth noting that syntax errors specifically are IMO vastly outbalanced in terms of human-hours by any other kind of errors. A bad map literal - you already get the exact line and column number with a clear description of what's wrong. Seconds. A function call with a wrong arity - tougher because often can only be handled in run-time, but still pretty easy. Seconds to minutes, if you trigger the error. Some other run-time error - the time can vary from a minute to a month or more, easily. Some strange AOT error - from hours to months as well.

p-himik14:05:53

> how to visually communicate this stuff as well as possible OK, this is intriguing. Right now we have text decorations (red underlining and similar), editor decoration (scroll bar decoration, line highlighting), separate panels (list of errors in the whole project, list of errors in the current file). How could this be improved, assuming we can get past the plain text somehow?

Rob Haisfield14:05:08

The error messages are really hard to parse for beginners so we get stuck on them for much longer than you might. I’m not sure if a structural editor would be useful for a l33t coder but I have little doubt that something that could prevent things that literally make no sense given the structure of the code or prompt you to input the right amount of arguments would be helpful to beginners

Rob Haisfield14:05:35

Re ways to make structure and rules more visually apparent, a (possibly) silly idea; https://clojurians.slack.com/archives/C03RZGPG3/p1622147058166500

Ed14:05:19

> For example, a functionality to compare the code that has been evaluated most recently with the current code. > Yes! Would show me if function definitions and values are out of date (edited) Doesn't cider do this currently? If a defn is evaluated it has a green highlight for me in the gutter on the left and that goes away if I change the code, until I load the buffer.

Ed14:05:54

I've often thought that it would good to have a query language for my codebase (kinda like https://github.com/borkdude/grasp maybe??) integrated into an editor - so maybe you could query for all the functions called by a particular stack trace, see them all on one view and edit them together, even though they may be distributed across multiple namespaces possibly even in multiple git repos ...

Ed14:05:25

but I'm not sure that's the sort of thing you're after, is it... I think if you'll be fighting an uphill battle trying to replace the immediate interface that someone uses to work with code ... the text based thing has worked very well for a long time, and visualisation graphical editing environments have repeatedly appeared and vanished without a trace ... although https://www.unisonweb.org/ looks like an interesting approach that may lend itself to changing that in the very long term??

Endre Bakken Stovner14:05:49

I agree with @U0P0TMEFJ about text based editing. A different approach would be to let users see your visualization of the code live in a browser while they are using their favorite text editor. This would not show all intermediate steps though, just when the user saved their file (you could have a program that sent the changes to the browser upon filechange).

quoll17:05:14

This has been an interesting thread, in that I wrote a project 5 years ago that parsed Clojure code and inserted it into Datomic for querying

quoll17:05:37

It was more POC than anything, but it worked 🙂

Rob Haisfield16:05:19

Woah cool! @U051N6TTC did you do anything with it? Is there anything you explored doing with it?

quoll17:05:48

Not for a long while, sorry! You’ll see my last checkin was 5 years ago.

quoll17:05:40

It’s a straightforward representation of the code. I imported the LispReader and then tweaked it for what I wanted. Specifically, I removed the macro evaluation step, and I kept comments.

quoll17:05:11

The lists are then all stored as linked lists in the db

quoll19:05:34

From memory, different data types (vectors, sets, maps) are all stored as lists with an attribute to say what type they are

Ed09:06:13

there's this too : https://github.com/Datomic/codeq ... which I've never played with enough to have anything substantial to say about it .... but it's interesting

Rob Haisfield17:06:08

@U051N6TTC Do you have any example databases that would be good to explore, perhaps through @U1G869VNV’s #portal?

quoll17:06:54

Explore for what? For code? (quick answer: no)

Rob Haisfield17:06:24

I guess I just wanted to have a concrete syntax tree of code to play around with, but if your GitHub link still works, could I try it?

rende1114:05:26

Hello! How to manage such a thing, monitoring, health checks? 

(def thread
    (Thread. (fn []
               (loop []
                 (println "Do some work!")
                 (recur)))))

  (.start thread)
In a real project, it is a consumer for the queue. Any good approaches, libraries for this? I already have a couple of those threads started from the main thread.

emccue15:05:16

@rende11 That is actually a fantastic question that I do not know the correct approach to

emccue15:05:15

Some potential ideas -

emccue15:05:41

use an executor service and a thread factory to make threads that have names that are relevant

emccue15:05:02

(Executors/newSingleThreadExecutor
  (-> (BasicThreadFactory$Builder.)
      (.namingPattern "chat-subsystem-%s")
      (.build)))

Alex Miller (Clojure team)15:05:26

or just use future if you want simpler code

Alex Miller (Clojure team)15:05:46

you will almost always want some way to tell that thread to stop

Alex Miller (Clojure team)15:05:31

closing over an atom that holds a boolean is one such way. if using an executor, it has shutdown methods

emccue15:05:32

with a named thread you can get decent monitoring with JFR

emccue15:05:45

or whatever other JVM monitoring tool

David Orme15:05:53

(.setName thread "queue consumer")

Alex Miller (Clojure team)15:05:57

you can change the name from inside the thread

Alex Miller (Clojure team)15:05:41

it's a little dirty but in a case like this where you're spinning a single thread, probably a good hack

emccue15:05:44

health checks you can inspect the executor, or have the tasks work on an atom when they crash

David Orme15:05:29

Then jconsole or jprofiler, or jstack.

David Orme15:05:21

Also, log from the thread and spelunk.

emccue15:05:35

https://gist.github.com/bowbahdoe/ce961e6c7449c1ffe9877082465b1f47 this is the closest i've done - from a personal thing a year-ish ago

ghadi15:05:41

naming a thread isn't going to help with monitoring -- you want something connected to the semantics of your domain

David Orme15:05:49

(Spelling on splunk?)

ghadi15:05:28

if your background threads are poll workers, exposing metrics on # of messages consumed -> prometheus or some metric system

emccue15:05:29

maintaining a reference to whatever stateful thing is potentially useful too

emccue15:05:49

^that is a better approach

David Orme15:05:55

Also, what Ghadi said and throw domain specifics in a map in an atom that you expose to whatever you use to monitor or debug prod issues.

emccue15:05:17

then set an alert on cloudwatch or wtvr for if the rate drops

emccue15:05:32

one thing i've done in a past job is pass through a "listener" I called whenever I did a key action in the consumer

emccue15:05:45

(twas in java, but same diff)

emccue15:05:20

(defn consumer-loop [listener]
  (let [msg           (poll-for-msg)
        _             (listener/started-processing-msg {:msg-id (:id msg)}]
    (try 
      (do 
        (insert-into-db (f msg))
        (listener/successfully-inserted-msg {:msg-id (:id msg)}))
      (catch ExpectedException e
        (listener/failed-to-inser {:msg-id (:id msg)}))))

emccue15:05:36

so thats one approach to get your domain level info

lspector15:05:59

Anyone know of, or feel like making, a site that will auto-reformat (re-indent) a field full of Clojure code? My use case is teaching beginners using no-installation tools that are just missing this crucial feature (currently Replit). If they could copy/paste/reformat/copy/paste via a site that just does reformatting, we'd have a workable solution, even if it's a little cumbersome.

David Orme15:05:25

Does the code need to be executed/executable from within the site?

David Orme15:05:31

Must the code be Clojure or is Clojurescript fine also?

lspector16:05:10

I'm just thinking about reformating, not execution here. Execution could be fabulous too, but the goal here is just to have a formatting tool that could be used in conjunction with another solution for execution, like http://Replit.com. Clojurescript would be fine also, since the formatting is the same. I'm hoping for something like the standard reformatting capability in many IDEs like Cursive or Calva, etc.

Franklin17:05:23

are there any data structures and algorithm books that use clojure?

didibus17:05:02

Not exactly what you ask, but there is this: http://www.sicpdistilled.com/

Rob Haisfield18:05:25

Oh that’s brilliant thank you @U0K064KQV

James Carr18:05:40

👋 anyone using open telemetry with clojure?

R.A. Porter18:05:17

I did a little proof of concept work with it at my last job. I used this library: https://github.com/tendant/clj-telemetry

R.A. Porter18:05:06

Ended up writing this macro (along with tracer which was an atom and this dynamic var 

(def ^:dynamic *trace-parent* nil)

James Carr18:05:09

Ah and the java agent to capture traces by default?

R.A. Porter18:05:52

Nothing by default; anything I wanted traced, I'd wrap in the with-trace macro.

R.A. Porter18:05:11

Not sure it was the best approach, but it worked well enough for the PoC.

ghadi18:05:59

@U01J1KVQZEF IIRC the java agent only knows how to open traces for certain libraries

ghadi18:05:28

if your web server isn't on the handled list, that may not work for you

James Carr18:05:31

yeah it seems to capture most however.

ghadi18:05:38

at cognitect we built a trace library for a client. Wasn't OTel compliant, we emitted things to Honeycomb

James Carr19:05:11

yeah maybe we should just give the native DD agent a try

James Carr19:05:24

but I think we are likely going to need to wrap things regardless.

James Carr19:05:47

for example, by default we see jetty calls, but pedestal and the like are missing

ghadi19:05:28

you'll need some sort of integration somewhere

ghadi19:05:56

we had a convenient pedestal interceptor that took a set of key paths (into the context) and automatically traced those into the active span

kenny19:05:49

We use the Java libs for opentracing and micrometer. Both are great. I think opentracing is moving to open telemetry? I don’t think we can switch because all the tooling hasn’t caught up yet (or something like that anyway).

ghadi19:05:36

yeah it's like the xkcd standards comic

ghadi19:05:47

endless churn in this space

ghadi19:05:43

none of the libraries we were using participated in open*anything, so making a tight 300LOC tracing library seemed like a low risk thing

ghadi19:05:55

spans are maps, that's pretty much it

javahippie13:05:49

After seeing this thread here yesterday, I had the time to write sth. about a wip on that topic: https://clojurians.slack.com/archives/C8NUSGWG6/p1622294885029000

dabrazhe22:05:03

I'd like to print in repl /save to file some of the keys from the list of maps I've made, see posted put in this slack thread. The idea is to align the keys under each other so that I can easily compare the values, like in a table. How can I achieve this? Eg with some sort of tabulation. 

{:BUY {:lastPrice 0.5 :volume 1 }   :SELL {:lastPrice 1.95 :volume 46}  :p2margin 17.35      :ticker "TSLA" }
{:BUY {:lastPrice 0.9 :volume 100 } :SELL {:lastPrice 1.95 :volume 460} :p2margin 18.2342420 :ticker "TSLA" }   
etc ...

dabrazhe22:05:21

({:BUY {:change 0.08000001
       :lastPrice 0.5
       :volume 1
       :etc :bunchotherkeys}
 :SELL {:lastPrice 1.95
        :safetyMargin 18.360813190331367
        :volume 46
        :etc :otherkeys}
 :p2margin 17.35
 :s-premium 1.27
 :ticker "TSLA"}
{:BUY {:change 0.08000001
       :lastPrice 0.9
       :volume 100
       :etc :bunchotherkeys}
 :SELL {:lastPrice 2.0
        :safetyMargin 18.360813190331367
        :volume 460
        :etc :otherkeys}
 :p2margin 18.2342420
 :s-premium 1.27
 :ticker "TSLA"})

dabrazhe22:05:55

cool. but only partially, because I'd need to reconstruct the maps. I'm looking for a way to extract only some keys from 2 level deep and then print a table