Fork me on GitHub
#clojure
<
2022-05-02
>
Nom Nom Mousse06:05:37

I'd like to make a declarative mini language to pick out the rows with specific values in a seq of maps. Let us take the oversimplified example:

[{:genome "hg19" :chip "H3K27me3"}
 {:genome "hg19" :chip "H3K4me3"}
 {:genome "hg19" :chip "Input"}}]
I'd like to separate the seq into two pieces. One for "Input" and one for the rest, call it ChIP. One could do something like {"Input" {:chip "Input"} "ChIP" {:chip {:not "Input"}} This should result in
{"Input" [{:genome "hg19" :chip "Input"}]
 "ChIP": [{:genome "hg19" :chip "H3K27me3"}
          {:genome "hg19" :chip "H3K4me3"}]}
In general I want to be able to split the seq into X pieces that together comprise the whole seq. Can you think of anything similar I could use for inspiration?

Nom Nom Mousse06:05:46

Is this something one of the #data-science libraries can handle declaratively?

delaguardo06:05:02

#meander could do that.

👍 1
Nom Nom Mousse06:05:07

True, also #specter I guess. I wasn't considering either because they are really made for knottier data structures.

Nom Nom Mousse06:05:11

They might be overkill for my simple problem, but I should study them for inspiration.

hiredman07:05:26

group-by

👍 1
Nom Nom Mousse11:05:16

That is a function, not data though 😄

igrishaev08:05:31

What is the deps.edn analogy for Lein's :injections ? Say, I'd like to require some of namespaces. That's how I do in in lein:

:injections [(require 'clojure.pprint)
                     (require 'clojure.inspector)
                     (require 'bogus.core)
                     #_(require 'hashp.core)]
How I can do the same with deps?

delaguardo10:05:04

I'm using project specific user.clj file to prepare development environment. Don't think there is a close analogy for :injections in deps.edn

igrishaev12:05:15

I wonder how to do that without changing the project. All the changes must be in my local deps.edn

igrishaev12:05:54

I tried to specify a local user.clj file but the project already has dev/user ns and it gets load instead of mine

rads18:07:48

Hey folks, I'm a big fan of hashp and I wanted to load it by default across all my clojure.tools.deps projects. Here's a tiny library I created to make this possible with minimal intrusion on existing code: https://github.com/rads/preload Not sure if this is the best way to do it, but I like that it doesn't have a dependency on a specific REPL API (in contrast to the solution in the https://insideclojure.org/2020/02/11/custom-repl/). GitHub issues are open if you have suggestions 🙂

Joshua Suskalo13:07:51

So this way of doing things gives you a way to load stuff into the user namespace.

Joshua Suskalo13:07:38

If you want to be really cheeky you could eval some code that defines some vars inside the clojure.core namespace before you load your own code, and those vars will be available from all namespaces afterwards.

Joshua Suskalo13:07:16

if you use potemkin then you could even do a potemkin refer-all to bring in a namespace to clojure.core which would make it available everywhere.

Joshua Suskalo13:07:43

critically though, this is only close to acceptable in local repl-driven development.

Joshua Suskalo13:07:47

It's absolutely not acceptable in production, and doing stuff like this (auto-requiring namespaces whenever a namespace is created) is dangerous because it can make it so your application works fine if you use things inside your namespace file that aren't required in code, which means your application won't work outside the development context.

rads23:07:19

Thanks for the tips 🙂 Especially the part about not doing this in production! The rads.preload library is intended for dev tool helpers only. Anything beyond that should be explicitly added to the project's deps.edn and/or source code and documented/tested as such For my own development, as an alternative to modifying clojure.core, I added a :local/root entry in the :extra-deps of my :preload alias and a preload entry for - . This one-character namespace has aliases for my commonly used dev functions For example, I have an atom called -/t which gives me the last 100 tap> results when I call @-/t . With the rads.preload library, I can add - as another namespace to load and the @-/t expression is available in all my projects

Nom Nom Mousse12:05:00

I have a Clojure app. When the app is running I want to be able to evaluate files with valid Clojure functions and put them in a specific namespace, say defs. Let us say that I have the form '(defn hiya [] (print "hi")). Is there a way to evaluate in a namespace defs so it is callable from anywhere as defs/hiya?

Clojure 1.10.3
user=> (in-ns 'defs)
#object[clojure.lang.Namespace 0x6f012914 "defs"]
defs=> (def f '(defn hoo [] (print "hi")))
#'defs/f
defs=> (clojure.core/eval f)
Syntax error compiling at (REPL:1:9).
Unable to resolve symbol: defn in this context
The following works, but I want to avoid having to prefix everything with clojure.core/:
defs=> (def f '(clojure.core/defn hoo [] (clojure.core/print "hi")))
#'defs/f
defs=> (clojure.core/eval f)
#'defs/hoo
defs=> (in-ns 'user)
#object[clojure.lang.Namespace 0x64bfd6fd "user"]
user=> (defs/hoo)
hinil

Nom Nom Mousse12:05:17

(in-ns 'defs)
(clojure.core/use 'clojure.core)
works

p-himik12:05:55

Instead of in-ns use ns - then clojure.core will be used automatically.

🙏 1
Andreas S.13:05:46

Hi! Im trying to execute the following in a REPL: ((java.time.LocalDateTime/now) (.minusHours 1)) but it seems java.time.LocalDateTime/now gets truncated to Long on which ther eis no .minusHours method I could I make this call work?

Martin Půda13:05:18

(.minusHours (java.time.LocalDateTime/now) 1)
=> #object[java.time.LocalDateTime 0x46a8689 "2022-05-02T14:16:15.340366900"]

(-> (java.time.LocalDateTime/now)
    (.minusHours 1))
=> #object[java.time.LocalDateTime 0x3c1cec05 "2022-05-02T14:16:37.845970"]

Martin Půda13:05:14

(doto (java.time.LocalDateTime/now)
  (.minusHours 1))

=> #object[java.time.LocalDateTime 0x243b6d46 "2022-05-02T15:18:26.814173600"]

Nom Nom Mousse13:05:25

I may want to define something in another namespace:

(def f '(defn hi/yo [] (println "yo")))
Here the ns is hi and the function is yo. However, I think I need to use:
(create-ns 'hi)
(intern 'hi 'yo (fn [] (println "yo"))
Is there a better way to extract the namespace and the function name from f than
(-> (second f) str (clojure.string/split #"/"))
["hi" "yo"]

Nom Nom Mousse13:05:46

Or is there a way to automatically define function in another namespace?

p-himik13:05:21

There are name and namespace functions. If you need a vector, you can just juxt:

=> ((juxt namespace name) 'a/b)
["a" "b"]

👍 1
p-himik13:05:50

Depending on what you're doing, you should be able to switch to that ns temporarily with in-ns, define the function, and then switch back with another in-ns. But that's bordering on being unreasonable, at least from my perspective. I would be very cautious around such code.

Ian Fernandez14:05:57

(let [p (promise)]
        (clojure.core.async/thread
          (deliver p many-IOs-exec-that-makes-swap-on-a-atom))
        (deref p timeout-value :error-timeout))
I've seen this pattern around some code, is this make any sense?! 😢

emccue16:05:47

I mean, it does make sense

emccue16:05:10

idk if that answers your "core" questions though

Ian Fernandez16:05:44

Why would I use async/thread within a promise?

Ian Fernandez16:05:37

AFAIK swap!’s are synchronous

emccue16:05:16

swaps are synchronous

emccue16:05:08

so if it just goes

(let [p promise]
  (thread (CODE THAT ONLY DOES BLOCKING OPS) (deliver p value))
  (deref p :timeout-value)) 

emccue16:05:34

then the only benefit is that you can decide to ignore the result of the thread if those blocking ops take too long

emccue16:05:02

otherwise just doing it inline without the promise or the thread makes more sense

Ian Fernandez16:05:55

Even so, blocking ops on another thread for an atom inside a reduce around for this (the function that call this one has a reduce) kills the propose of this entirely

Ian Fernandez16:05:15

It would be synchronized with the atom

emccue16:05:26

atom ops aren't synchronized

emccue16:05:18

they are synchronous, and having multiple threads working on the same atom at the same time can cause contention and retries, but work will be done in parallel

1
Eugen14:05:16

Is there a way to have an implementation that delegates to default value for annotation? I have this annotation: https://github.com/apache/directory-server/blob/272c7c3686e8177f4f056983d294205bfa1a46c5/server-annotations/src/main/java/org/apache/directory/server/annotations/CreateLdapServer.java And this code:

(let [x (reify org.apache.directory.server.annotations.CreateLdapServer
            (name [this] "my-ldap"))]
    (println (.name x))
    (println (.saslPrincipal x))))
when evaluated, I get this:
clj꞉ieugen.ldap-auth-provider.core꞉> 
my-ldap
; Execution error (AbstractMethodError) at ieugen.ldap-auth-provider.core/eval11686 (REPL:66).
; Receiver class ieugen.ldap_auth_provider.core$eval11686$reify__11687 does not define or inherit an implementation of the resolved method 'abstract java.lang.String saslPrincipal()' of interface org.apache.directory.server.annotations.CreateLdapServer.
Implementing name gives me the right result. But the annotation has default values - so I should be able to get those if I don't do anything. Right ?

borkdude14:05:12

Why does Java not use file:// but file: - some other tools expect file://:

$ bb -e '(str (.toURI (io/file ".")))'
"file:/Users/borkdude/dev/clerk/./"

borkdude15:05:15

@U07FP7QJ0 Do you know? Since you're kind of a URI expert?

p-himik15:05:47

The relevant part of the RFC:

file-URI       = file-scheme ":" file-hier-part

file-scheme    = "file"

file-hier-part = ( "//" auth-path )
                     / local-path

ericdallo15:05:36

we had lots of issues on clojure-lsp regarding that, but I can recall why we now always use 3 slashes file:///path/to/file.clj @U016JRE24NL do you recall why we needed that with 3 slashes instead of 1?

ericdallo15:05:26

> All things being equal, the triple slash and "localhost" keyword only exist to ensure conformance with valid URI/URL syntax. https://superuser.com/a/971837 So it seems that 3 slashes are valid URI and URL while one slash is only a valid URI

borkdude15:05:58

$ bb -e '(str (.toURL (.toURI (io/file "."))))'
"file:/Users/borkdude/dev/clerk/./"

1
Luis Thiam-Nye20:05:50

@UKFSJSM38 I can spot https://github.com/clojure-lsp/clojure-lsp/blob/42f88f7fa8f3d64dc51ccfefb16a92731406afea/lib/src/clojure_lsp/shared.clj#L124 // prefix. Using this style means you have a uniform syntax between different schemes (like jar:file, zipfile:) and makes parsing the strings less of a headache. I am not sure if it has any other significance for the project though. To avoid any creative URI formats, clojure-lsp first converts the file to a Path then to a URI:

user=> (-> "." io/file .toPath .toUri str)
"file:///C:/Users/luist/./"
as opposed to:
user=> (-> "." io/file .toURI str)
"file:/C:/Users/luist/./"

ericdallo21:05:07

I see, I'm asking because clj-kondo is using URIs now and it's returning URIs with only one slash, I was wondering why it differs from clojure-lsp, since it's different we can use it yet

Luis Thiam-Nye10:05:10

I suppose an easy solution would be to do (-> x uri->filename filename->uri)

ericdallo11:05:40

Yeah, I was trying to avoid the need of normalization uris from kondo, so the problem with one slash is zipfile and jar:file only?

borkdude11:05:36

I'd be happy to "normalize" on the clj-kondo end of things, I'm just curious why Java behaves like it does right now

ericdallo11:05:04

Yeah, me too

Luis Thiam-Nye12:05:25

The // is a part of the URI's optional authority component. When the URI has no authority, the // can be omitted. It is referred to as 'redundant syntax' https://docs.oracle.com/javase/8/docs/api/java/net/URI.html, so maybe it is better to use the single / and clj-kondo is probably fine how it is. So jar:file and zipfile could also presumably omit the //, and conform-uri could be adjusted accordingly. I do not know if there would be any further implications — but the fact that the same location can be expressed in different ways is something to be careful about. Perhaps a concern would be if lsp clients did not know how to handle the single slash, but that would be their problem. The main problem in the past with clojure-lsp and URIs was largely about the encoding of the paths, not so much the slashes.

ericdallo12:05:09

I see, thanks for the explanation, so my only concern now is if clients sends URIs with one slash or something, we would use one slash on kondo/lsp internals but would need to conform the URIs that come from clients to be one slash, right?

Luis Thiam-Nye13:05:44

I remember that clojure-lsp https://github.com/clojure-lsp/clojure-lsp/blob/42f88f7fa8f3d64dc51ccfefb16a92731406afea/lib/src/clojure_lsp/crawler.clj#L128 and uses those when converting a file path to a URI, and the same thing could be done for the // if found to be necessary. It would be an issue if clj-kondo returns URIs that do not match the style of the client. So, you could conform all client URIs to a known style (and possibly convert back in the responses), but I think it would be easier to conform clj-kondo's URIs into the style used by the client.

ericdallo13:05:59

yeah, those are valid possibilities indeed, probably conform client's URI to a single slash would be the best one IMO

Luis Thiam-Nye13:05:50

You may also need to make sure that URIs use consistent drive letter capitalisation and colon encoding, not just the number of slashes. And if this means responding to the client with a URI that is encoded differently to how it was initially sent in the request, you would have to hope that that is not a problem.

borkdude13:05:14

So what I gather from this is that clj-kondo's format, isn't incorrect, right?

Luis Thiam-Nye13:05:25

It is valid for clj-kondo to use a single slash

borkdude13:05:43

Then I'll just let the other tools deal with their problems, thanks :)

👍 1
ericdallo13:05:16

yeah, I can't see we easily using clj-kondo's uri on clojure-lsp without checking each of these issues @U016JRE24NL mentioned, so for now I think we will need to keep using fileanme until there 😔

borkdude13:05:53

fine with me :)

👍 1
vncz14:05:25

How do you people call the key+value pair in a map?

vncz14:05:28

Property or Attribute?

mpenet14:05:27

a map-entry

devn14:05:43

technically it's a map entry

👍 1
vncz14:05:57

Hmm ok. I am trying to be more abstract

vncz14:05:10

A map entry, an element in a vector, an element in a set

noisesmith14:05:52

"thing" or "object"

vncz14:05:47

Not quite what I am looking for

noisesmith14:05:05

element of a collection

devn14:05:20

element seems fine

devn14:05:26

honestly I've always referred to it as a kv pair

4
vncz15:05:00

Ok let’s rephrase the question then. In your mind, what is the definition of attribute, and what is the definition of a property?

timothypratley15:05:46

These are both "naughty words" from OOP 😛

timothypratley15:05:02

We speak not of them.

👍 2
timothypratley15:05:23

So I'd like to answer your question as attribute is the building block of a Datom/Datalog database. In a triple store, the word attribute makes a lot of sense. It should be a fully qualified keyword that stands alone in defining a domain of values that can be associated with an entity. A property is an interop necessity. <= IMO this would be uncontroversial definitions in this channel, but very unusual answers in the broader discourse. Sorry if this is off topic... not sure what your ultimate goal is so just jumping in with my thoughts 🙂

👍 1
Joshua Suskalo15:05:22

I'd agree, these are likely relatively uncontroversial definitions here.

devn15:05:46

to Tim's comment, it's hard for me to be helpful without knowing what you're trying to accomplish. Are you naming something in code? Is this for technical writing? Educational content? Who's the audience? etc.

vncz15:05:06

Maybe that is a better question to ask

noisesmith15:05:16

I don't use either of those terms to describe clojure data

☝️ 6
vncz02:05:15

Agreed, I do not either. I was more looking for some alternatives way to express that concept - in particular in API Design :)

kwladyka16:05:42

(type (first {:a 1 :b 2}))
=> clojure.lang.MapEntry
source of the name for key-value pair

kwladyka16:05:27

I mean this is why map entry explained in code

icemanmelting17:05:37

Guys, quick macro related question, is there a way to use a macro scope defined argument in the body that we want to have evaluated inside the macro? Something like this:

(defmacro test [expr]
  (let [a-value# (inc 1)]
    ~expr))

(test (do (prn ~'a-value#)))

Joshua Suskalo17:05:51

You would need to make the value available under a normal symbol. So something like this:

(defmacro test [expr]
  `(let [~'a-value (inc 1)]
     ~expr))

(test (do (prn a-value)))

p-himik17:05:34

As a better alternative - use what as-> uses and give users a way to provide their own symbol.

Joshua Suskalo17:05:41

this is desirable in many cases, but there are some valid cases where breaking hygiene is the point of the macro.

icemanmelting17:05:44

@suskeyhose the solution you provided, just throws an error when expanding

Joshua Suskalo17:05:57

I'd love to see the error here, as it works fine from my repl

icemanmelting17:05:13

Syntax error macroexpanding clojure.core/let at (/private/var/folders/ks/08g0w0x15915ntgmfbwh8w4h0000gn/T/form-init817900795893979501.clj:2:3). (clojure.core/unquote (quote a-value)) - failed: simple-symbol? at: [:bindings :form :local-symbol] spec: :clojure.core.specs.alpha/local-name (clojure.core/unquote (quote a-value)) - failed: vector? at: [:bindings :form :seq-destructure] spec: :clojure.core.specs.alpha/seq-binding-form (clojure.core/unquote (quote a-value)) - failed: map? at: [:bindings :form :map-destructure] spec: :clojure.core.specs.alpha/map-bindings (clojure.core/unquote (quote a-value)) - failed: map? at: [:bindings :form :map-destructure] spec: :clojure.core.specs.alpha/map-special-binding

icemanmelting17:05:19

sorry about that

Joshua Suskalo17:05:19

looks like you missed the vector around it? I'm unsure, but this is an error in transcribing what I wrote.

icemanmelting17:05:24

I missed the back quote

Joshua Suskalo17:05:29

yeah, that's important

icemanmelting17:05:45

I had it there for sure, but for some reason I deleted it by mistake

icemanmelting17:05:52

it works now, thanks a lot for your help!

icemanmelting17:05:06

I will check the as-> macro @p-himik thanks

seancorfield17:05:44

🧵 please...

👏 1
icemanmelting17:05:25

Joshua’s solution worked, I was missing the back quote

icemanmelting17:05:34

thanks for everyone’s help

fadrian17:05:09

I have a routine using core.async defined as follows:

(defn get-checked-items []
  (let [items (all-items)
        chan (a/chan max-threads)
        results (atom [])]

    ;worker threads
    (dotimes [_ max-threads]
      (a/go-loop [item (a/<! chan)]
        (let [cp (check-item item) ;returns nil when no results
              _ (when cp (swap! results conj cp))
          (when item (recur (a/<! chan))))))

    ;item addition thread
    (a/go
      (doseq [item items]
        (a/>! chan item))
      (a/close! chan))

    @results))
This seems to work fine, except that the routine returns immediately (when @results is still empty) and then continues working in the background until all items are exhausted. Is there any way to wait on the completion of all worker threads? Is there a more idiomatic way of dealing with this kind of situation?

hiredman17:05:22

Collect all the go blocks you create and wait for them to exit

hiredman17:05:34

You may want to look at using one of the pipeline variants

hiredman17:05:02

your item addition thread is just onto or whatever it is called now

hiredman17:05:28

a difference with a pipeline variant is those will be ordered

fadrian17:05:42

So rather than the code I have now, you suggest I use something like this:

(let [concurrent max-threads
      output-chan (chan)
      input-coll (all-items)]
  (pipeline-blocking concurrent 
                     output-chan 
                     (map check-item) 
                     (to-chan input-coll))
  (<!! (a/into [] output-chan)))
where check-item is made as a transducer function? Correct?

devn17:05:32

Is anyone aware of an explain-like tool for cl-format strings?

timothypratley18:05:48

He's the only person I've met who "gets it" 🙂

devn18:05:57

i feel like i’ve mostly worked it out at this point

👍 1
devn18:05:21

my initial attempt is something like ~<:> is a logical block, within which ~; is a command separator which separates the prefix, body, and suffix. In this case #( is the prefix and ) is the suffix. ~@{} means iteration, and within the iteration block we have ~w which means print the argument from a list of things. ~^ means to terminate if we run out of things in the list of things we’re printing. then we have a space. ~_ says to print a newline if we’re over the limit of characters for this line.

🤯 2
enn19:05:49

😆 4
💯 1
devn17:05:49

"~<#(~;~@{~w~^ ~_~}~;)~:>" is baking my noodle.

devn17:05:00

(for context, i discovered a bug a long time ago in pprint’s code-dispatch for (fn* [x] x) -like things, and figured I’d take a stab at actually fixing it)

rgm23:05:44

I'm hunting for possible libraries or techniques for a kind of task that almost certainly has a googleable name, but I don't know it: I have plaintext e-mails coming in and want to act on their message bodies. They're not exactly parseable (ie. I don't think a grammar exists, so it doesn't feel like instaparse would work) and they'd be tedious to regex, though I suppose that might eventually work. Some parts of the body are (probably accidentally) TSV as a result of doing layout in plaintext. I guess I'm after ... some data-defined way to establish zones relative to consistent landmarks, and say, ok, use this regex on the line 2 lines later and give me the groups. I can imagine doing a bunch of this with drop-until landmark, etc. working at the level of a line-seq. But I'm wondering if there are good approaches for almost-structured data that operates above lines and regexes and seqs and below formal grammars and parsing.

hiredman23:05:55

a state machine

hiredman23:05:05

(reduce (fn [[state-name data] line] (case state-name ..do stuff with line and data based on the content of line and the value of state-name and return a possibly new state...)) [:init {}] lines)

hiredman23:05:41

like if you are in :init and see a blank line then move to :after-email-headers or whatever, if you are in :after-email-headers and you see the columns headers for the tsv then move to :accumulate-tsv, and in :accumulate-tsv add the lines to data until a line is blank

rgm04:05:42

oh right, thanks. Forgot about the idea of going SAX-style.

vncz02:05:15

Agreed, I do not either. I was more looking for some alternatives way to express that concept - in particular in API Design :)

rads18:07:48

Hey folks, I'm a big fan of hashp and I wanted to load it by default across all my clojure.tools.deps projects. Here's a tiny library I created to make this possible with minimal intrusion on existing code: https://github.com/rads/preload Not sure if this is the best way to do it, but I like that it doesn't have a dependency on a specific REPL API (in contrast to the solution in the https://insideclojure.org/2020/02/11/custom-repl/). GitHub issues are open if you have suggestions 🙂