Fork me on GitHub
#clojure
<
2021-11-10
>
mbertheau10:11:50

In an iterative process I have source, process and sink. With transducers I specify just process. Source and sink are abstracted. With reduce I specify process and sink. Only source is abstracted. Is there something that allows me to specify source and process, but not the sink? I.e. I have a non-standard source of items, but I don't want to specify the sink part. An example would be a process that combines two different sources into one, sometimes taking an item from one source and sometimes from the other, and then passing the item on to the next step in the process, without specifying what that step is.

Ben Sless10:11:48

An Iterable?

jaihindhreddy10:11:30

eduction might be what you are looking for.

☝️ 2
Ben Sless10:11:55

I wonder how Eduction will fit into this given that it "reruns" every time, and I'm not sure how you'll "switch" or interleave between two eductions

1
mbertheau10:11:20

Let me look that up, thanks!

jumar14:11:44

There’s also sequence if you want to cache stuff (unlike eduction)

Sathiya11:11:24

Hi. I have a need to dynamically choose the def in various name spaces based on conditions. Assume i have three namespaces foo, bar and baz and they all have a (def name "i am foo/bar/baz") . I need to access the the def in the namespace dynamically based on condition. like (defn your-name [s] (<find-namespace>/name)) . If this was a function i was able to do it with the help of resolve and symbol . But unable to access def in the same manner. Has anyone tried something similar or can help with this. Thanks in advance

p-himik11:11:55

Clojure 1.10.3
user=> (ns a)
nil
a=> (def x 1)
#'a/x
a=> (ns b)
nil
b=> (def x 2)
#'b/x
b=> (in-ns 'user)
#object[clojure.lang.Namespace 0x6f6962ba "user"]
user=> (require 'a 'b)
nil
user=> @(ns-resolve 'a 'x)
1
user=> @(ns-resolve 'b 'x)
2

Sathiya12:11:04

Wow. Thank you @U2FRKM4TW 🙂

👍 1
seancorfield16:11:17

If you're on 1.10 or later, you can use requiring-resolve to avoid the need to require the two nses:

Clojure 1.10.3
user=> (ns a)
nil
a=> (def x 1)
#'a/x
a=> (ns b)
nil
b=> (def x 2)
#'b/x
b=> (in-ns 'user)
#object[clojure.lang.Namespace 0x3ff57625 "user"]
user=> @(requiring-resolve 'a/x)
1
user=> @(requiring-resolve 'b/x)
2
user=> 

👍 1
seancorfield16:11:51

(`requiring-resolve` is also thread-safe at runtime which plain require is not, I believe)

dpsutton15:11:40

does this make anyone else uncomfortable?

(defmuli foo [event & _] event)
(defmethod foo :create-user [event user-id])
(defmethod foo :invite [event user-id date])
I’m not a fan and I’m trying to determine if its just superficial style or if its really not a great pattern.

pavlosmelissinos16:11:39

If you're worried about the arity mismatch, you could use a map instead of a variadic signature

dpsutton16:11:11

yes that’s my gut reaction but i’m trying to figure out if its just stylistic or an actual concern

pavlosmelissinos16:11:29

I think it comes down to personal choice but fwiw I'd use maps

dpsutton16:11:11

agree. but if its personal choice i won’t mention it in the PR review. Thanks

Ben Sless16:11:52

At that point I'd rather the arguments were in a map

JohnJ16:11:19

does event and user-id can come as a map or vector or json, etc...?

p-himik16:11:12

It would become rather comfortable for me if foo had a name that would clearly signal that the first argument defines the rest.

FiVo16:11:51

From just the snippet I would definitely change it to two defn 's with names that clearly signal their purpose

dpsutton16:11:00

i guess its really just a variadic multimethod where different implementations have different arities

dpsutton16:11:51

there are many more, just a minimal snippet that shows the pattern. multimethod that takes different number of parameters for different bodies. It feels funny but i’m not sure if it should

p-himik16:11:49

How does this edit feel?

(defmulti foo [event & _args] event)

(defmethod foo :create-user [event & args]
  (assert (= (count args) 1))
  (let [[user-id] args]
    ...)

(defmethod foo :invite [event & args]
  (assert (= (count args) 2))
  (let [[user-id date] args]
    ...)
Now all methods have the same arity. This should be completely identical (barring the exception thrown in case the argument number mismatch) to your example. Not saying that this is how it should be done - just trying to explore the space.

dpsutton16:11:36

i don’t like that solution but i think that might highlight why it feels so weird to me. so its a good exploration

dpsutton16:11:19

like that makes the code worse, but it makes it more honest. and that honesty to what is going on is what i don’t like about the solution

👍 1
Colin P. Hill16:11:48

another option might be to wrap the variadic part up into a single parameter

(defmuli foo [event args] event)
(defmethod foo :create-user [event [user-id]])
(defmethod foo :invite [event [user-id date]])
Or even, to borrow a page from re-frame's book, go a step further and make the entire event, with its parameters, a single data structure – which could just be a vector
(defmuli foo [event] first)
(defmethod foo :create-user [[event-id user-id]])
(defmethod foo :invite [[event-id user-id date]])

Colin P. Hill16:11:25

it feels to me like "an event" is a single thing and passing it around in pieces – its ID here, its parameters there – doesn't quite line up semantically

dpsutton16:11:26

I think i would just go to maps. It just feels weird for it to be structurally different (ie, number of args). Maps with optional keys is a well-explored and comfortable space

👍 2
p-himik16:11:56

Re-frame is moving in the same direction, I believe.

Colin P. Hill16:11:48

yeah, I don't know that I would necessarily advocate a vector as the structure of choice (it was just easy to mock up), just advocating the idea of putting it all in some singular structure

👍 1
emccue18:11:06

This might be silly, but why not just

(defn- foo-create-user [{:keys [user-id]}]
  …)

(defn- foo-invite [{:keys [user-id date]}]
  …)

(defn foo [event data]
  (condp = event
    :create-user (foo-create-user data)
    :invite      (foo-invite data)))
Multimethods are good, but if you don’t have a reason to have dispatch be extensible, ifs are still okay

dpsutton18:11:11

i kinda agree with you. it’s internal and not really open dispatch

ivana19:11:01

Yesterday we found another reason for wrapping many sequential parameters into one structure (vector, map etc) - limitation of datomic db-functions with 10 arguments max 🙂 So, multi-sequential-arguments functions seems to look like stone age of programming and C language. Also they creates another problem - the order of arguments, followed by partial-curryng, existance of flip in Haskell and so on and so forth

lispers-anonymous14:11:00

> Re-frame is moving in the same direction, I believe. @U2FRKM4TW what makes you say that? I would very much like for re-frame to move in that direction with subscriptions and events

p-himik14:11:01

And if you prefer to use maps over vectors - you can do that already! :) Just make sure that your own code always uses only 2-element vectors where the first item is the event/sub ID and the second item is the arguments. You can even write your own dispatch and subscribe that do that for you.

lispers-anonymous14:11:28

That is great news, I'll spend some time reviewing that issue. I have been working on a library for my job written on top of re-frame that adds some functionality to subscriptions and events (I am trying to see if we can open source it, but have to make sure the company will allow it). It has been difficult to work with the positional nature of passing arguments in a vector, since part of the library involves automatically injecting arguments into the vectors. Events are easy enough to deal with using interceptors. Subscriptions are another story. I have it all working by appending the arguments to the end of query vectors but I would feel a lot better about it if I had a map to work with.

Colin P. Hill14:11:05

Always nice to see more libraries take full advantage of the "throw everything in one map with namespaced keys" pattern

catjam 1
JoelK17:11:23

Hello, I have question for anyone who may be familiar with compojure-api. I have an api that I would like to document with swagger ui but I would prefer that the swagger documentation only be accessible to those who have been previously authenticated and otherwise redirect to the website’s login page. In compojure-api is there a way to add authentication verification middleware specifically to the swagger ui pages (e.g. to check whether the session corresponds to a logged in user)? Simple example:

(def my-api
  (sweet/api
   {:swagger
    {:ui "/swagger"
     :spec "/swagger.json"
     :data {:info {:title "simple" :description "example"}}}}
   (sweet/context "/api" []
     (sweet/GET "/okay" []
       :return s/Str
       "OK"))))
Note in the above example, GET /api/okay does not require authentication so my-api cannot be wrapped in an authentication middleware.

kwladyka18:11:25

Do you recommend any stock market library to use in Clojure? something what I can use to count when and what sell / buy with which price. I am not exactly sure what functionality I am looking. I started to think about this today. Do we have good library for that purpose? (Clojure or Java)

vlaaad18:11:42

Oh I'd love to see a library that can count when and what to sell/buy

vlaaad18:11:26

Free money :D

kwladyka19:11:12

I mean the main algorithm should be made by me, but I am sure there are some libraries which support such operations on stock market. And for sure this libraries implement common algorithms, statistics etc.

kwladyka19:11:28

There should be some libraries which I can use for this purpose, but I don’t know yet I want to use them 😉

vlaaad19:11:52

so something like backtesting and algorithms for technical indicators (e.g. macd, vwap)?

vlaaad19:11:06

that would be interesting! please ping me if you find it!

kwladyka19:11:29

sure, I will

phronmophobic21:11:30

I'm not familiar enough with space to make a recommendation, but I do remember seeing https://www.reddit.com/r/Clojure/comments/cpf0xh/code_review_wanted_for_my_stock_broker_api/ by @U064UGEUQ

👍 1
jjttjj16:11:55

Yeah my project was for interfacing with Interactive Brokers (a particular stock broker) @UCSJVFV35 has some work on doing more trading analysis in clojure here: https://github.com/clojure-quant/trateg

kwladyka11:11:50

thanks, I will check this one too

grounded_sage20:11:50

I tried getting that one running @U064UGEUQ but wasn't successful. I am also interested @U0WL6FA77. Was just talking to another Clojurian about this stuff and how it seems like the backtesting infrastructure would be good to have shared.

grounded_sage21:11:09

Though I am particularly interested in crypto markets. I head that it might be difficult to have backtesting shared between crypto and traditional markets and most people focus on traditional

kwladyka21:11:47

> crypto markets yes, this is what I am interested in too. Although I found interesting environment for some tests: eve online - this is a MMO game. But the math should be very similar. Of course effective algorithm is very individual thing. So I am thinking about make first simple ver. as a proof of concept for this game and next do it for cryptocurrency.

kwladyka21:11:22

I know it sounds a little crazy, but it can make sense. We will see.

awb9911:11:55

What is interested about this library is that it uses tech ml dataset for very efficient handling of tables, and Goldy for visualisation in the browser. The library is alpha, but it is already quite powerful.

kwladyka16:01:30

Do you know good community (forum, slack, discord etc.) for software developing for cryptocurrency trading?

Sam Ritchie19:11:01

Is there some way to get a reify object to NOT use its metadata when calculating equality? For example, a vector also implements IObj but ignores its metadata for hashcode purposes, so it’s fine to use these as keys in a map.

Ben Sless19:11:07

You'll have to implement equals yourself An object with meta is no the same object instance. The default equals implementation is identity

Sam Ritchie19:11:24

yes, that makes sense. any recommendations on the best way to do it, to say “identity of the thing wrapped in metadata”?

Sam Ritchie19:11:58

I guess I can just generate a UUID and use that, for this purpose

hiredman19:11:09

or switch to a defrecord

☝️ 1
Sam Ritchie19:11:55

okay, nice, that is great.

Alex Miller (Clojure team)19:11:31

roll back - what does the original question mean? metadata is never used for comparing equality

Alex Miller (Clojure team)19:11:07

do you really mean "reify" objects have identity equality semantics and adding/changing metadata changes identity?

Sam Ritchie19:11:11

@U064X3EF3 yeah, take a look at

(defprotocol ICake (cf [x]))

(let [a (reify ICake (cf [x] x))
      b [1 2]]
  ;; the resulting map has TWO entries:
  (= 2 (count (assoc {a 1} (with-meta a {:k "v"}) 2)))

  ;; a vector manages to not use its metadata in equality / hashcode:
  (= 1 (count (assoc {b 1} (with-meta b {:k "v"}) 2))))

Alex Miller (Clojure team)19:11:01

just want to clarify, that's not a special property of vectors, it's the default behavior - metadata does not affect equality or hashcode as a general principle

Alex Miller (Clojure team)19:11:50

reified objects extending from Object get Object behavior, which is based on identity

Sam Ritchie19:11:02

but it is affecting equality for the reify in this example, right?

Alex Miller (Clojure team)19:11:08

you could provide your own equals when you reify ICake that does something different

Sam Ritchie19:11:10

you are saying that is a bug, and it should work like vector

Sam Ritchie19:11:25

ah, identity vs equality, got it

Sam Ritchie20:11:01

I think the defrecord solution is the right one, I have gotten away with this funky reify thing long enough

Alex Miller (Clojure team)20:11:05

equality defaults to value-based in Clojure, but identity-based in Java. when you use reify you are tapping into interop and choosing that world

hiredman20:11:08

reified objects get the value metadata api not the identity metadata api, but have identity based equality by default

hiredman20:11:16

functions have this same mismatch

hiredman20:11:54

user=> (let [f (fn [])] (= f (with-meta f {})))
false
user=>

Alex Miller (Clojure team)20:11:23

using a record seems like a reasonable alternative, but it's but one option. I don't know all the dimensions to compare them on.

lilactown20:11:25

it could be cool to have an anonymous record macro a la reify

Alex Miller (Clojure team)20:11:04

for what? reify can close over locals

Alex Miller (Clojure team)20:11:56

what problem are you trying to solve that's not solved by records or reify

lilactown20:11:48

was thinking "`deftype` is to reify what defrecord is to ???"

Alex Miller (Clojure team)20:11:05

I don't understand the relationship from deftype to reify that you are trying analogize

lilactown20:11:53

a map-like that can also have custom protocol implementations that doesn't extend the protocols to the global map type

lilactown20:11:14

and those protocol impls can close over locals

lilactown20:11:52

it wouldn't be necessary to do this in clojure.core, could have a library build it on top of reify of course

lilactown20:11:57

this is not a feature request 😛 unless someone actually has a good reason for it. just was a thought.

lilactown20:11:13

maps could work if the protocol you're using supports implementation via metadata

Alex Miller (Clojure team)20:11:39

what do you imagine "map-like" but "doesn't extend to the global map type" to mean?

Alex Miller (Clojure team)20:11:20

you could mean "supports lookups" (ILookup) but is not a persistentmap (IPersistentMap / map?)

Alex Miller (Clojure team)20:11:29

I think in most cases using such a thing you would quickly want the other parts of maps / records in most cases

Alex Miller (Clojure team)20:11:15

we have a couple places where we've done things kind of like this in Java - TaggedLiteral, ReaderConditional or clojure (fspec comes to mind)

Alex Miller (Clojure team)20:11:54

implementing only ILookup is trivial. the gap between there and what defrecord provides is large.

Alex Miller (Clojure team)20:11:02

I am trying to tease this out a bit because there are times when I want the gross implementation guts of records but without records (and I'm sure this is in the ballpark of when people reach for Potemkin's various def-map-type, def-derived-map, def-abstract-type, deftype+ etc).

lilactown21:11:58

I think what I'm thinking of is solved by allowing your protocol to be extended via metadata

lilactown21:11:05

(defn postgres-component
  [config]
  ;; creates a new map-like object (i.e. allows get, assoc, dissoc, clojure equality semantics, seqable, etc.)
  (record config
    ;; extend some additional protocols
    IStart
    (-start [this]
      (assoc this :conn (create-connection config)))
    IStop
    (-stop [this]
      (close-connection! (:conn this)))
      (dissoc this :conn))))

lilactown21:11:02

I think having a parameterizable map constructor would also be cool, which I think is what things like def-map-type give. the ability to create something which, as you say, abstracts the gross guts of records but allows one to customize what happens on get assoc etc.

hiredman21:11:17

people have written such things, macros for custom map types, clj-http has a custom map type for headers making the keys case insensitive

lilactown21:11:40

yeah, I have several custom implementations of a map laying around my projects

Sam Ritchie19:11:39

(defprotocol ICake (cf [x]))

(let [a (reify ICake (cf [x] x))
      b [1 2]]
  ;; the resulting map has TWO entries:
  (= 2 (count (assoc {a 1} (with-meta a {:k "v"}) 2)))

  ;; a vector manages to not use its metadata in equality / hashcode:
  (= 1 (count (assoc {b 1} (with-meta b {:k "v"}) 2))))

hanDerPeder20:11:54

why isn’t namespaced vectors a thing?

Alex Miller (Clojure team)20:11:09

not ready to bite on that idea yet. we've talked about it off and on.

Alex Miller (Clojure team)20:11:09

(and just to stay clear with it, you mean namespaced vector syntax (neither maps nor vectors "have" a namespace))

Alex Miller (Clojure team)20:11:40

the next logical step is - what about sets. imagining #:account#{:name} made us wretch a bit and stop taking steps :)

😂 1
hanDerPeder20:11:49

I see your point. haven’t ever felt I wanted namespaced sets, but vectors pops up from time to time. (mostly datomic pull patterns and such)

hanDerPeder20:11:51

[:company/name
    :company/phone-number
    {:company/employees [:user/name
                         :user/email]}]
stuff like this

Alex Miller (Clojure team)20:11:16

it would also possibly support some interesting things in destructuring

Takis_21:11:35

Hello i am trying the bellow code, and sometimes it runs other times i get stackOverflow

Takis_21:11:50

(clojure.data.json/pprint [{"$set" {"array_1" {"$map" {"input" "$array_1", "in" {"$cond" [{"$ne" ["$$m1.id" 2]} "$$m1" {"$mergeObjects" ["$$m1" {"array_2" {"$let" {"vars" {"size_position" {"$reduce" {"input" "$$m1.array_2", "initialValue" [0 -1], "in" {"$let" {"vars" {"index_pos" "$$value", "m2" "$$this"}, "in" {"$let" {"vars" {"index" {"$arrayElemAt" ["$$index_pos" 0]}, "pos" {"$arrayElemAt" ["$$index_pos" 1]}}, "in" {"$cond" [{"$lt" [110 "$$m2.when"]} [{"$add" ["$$index" 1]} {"$add" ["$$index" 1]}] [{"$add" ["$$index" 1]} "$$pos"]]}}}}}}}}, "in" {"$let" {"vars" {"asize" {"$arrayElemAt" ["$$size_position" 0]}, "position" {"$arrayElemAt" ["$$size_position" 1]}}, "in" {"$let" {"vars" {"prv_position" {"$subtract" ["$$position" 1]}}, "in" {"$switch" {"branches" [{"case" {"$or" [{"$and" [{"$ne" ["$$position" -1]} {"$let" {"vars" {"m3" {"$arrayElemAt" ["$$m1.array_2" "$$prv_position"]}}, "in" {"$eq" ["$$m3.value" 43]}}}]} {"$and" [{"$eq" ["$$position" -1]} {"$let" {"vars" {"m3" {"$arrayElemAt" ["$$m1.array_2" 0]}}, "in" {"$eq" ["$$m3.value" 43]}}}]}]}, "then" "$$m1.array_2"} {"case" {"$ne" ["$$position" -1]}, "then" {"$concatArrays" [{"$slice" ["$$m1.array_2" 0 "$$position"]} [{"value" 43, "when" 50}] {"$slice" ["$$m1.array_2" "$$position" "$$asize"]}]}}], "default" {"$concatArrays" [[{"value" 43, "when" 110}] "$$m1.array_2"]}}}}}}}}}}]}]}, "as" "m1"}}}}])

Takis_21:11:17

its just a mongodb big query, i dont know which part causes it, but it doesn't happen always

Takis_21:11:23

if i restart the app, it either works always or never

Takis_21:11:35

i tried also https://github.com/dakrone/https://github.com/dakrone/cheshire which worked ok always, but the pprint result is nicer in clojure.data.json

Alex Miller (Clojure team)21:11:40

what version of data.json are you on?

Takis_21:11:12

i tried all version, it happened in all, so i kept the latest release "2.4.0"

Takis_21:11:26

not all versions, almost all

Takis_21:11:51

:clojure.main/message
 "Syntax error (StackOverflowError) compiling at (live_j.clj:157:1).\nnull\n"

Alex Miller (Clojure team)21:11:11

well that is certainly a long stack. I can reproduce it. I'll file a data.json ticket about this

Takis_21:11:02

it always worked when you tried it? for me sometimes works always, and other error (when i restart the app)

Alex Miller (Clojure team)21:11:54

well I tried it once and it failed :) I think you're just getting lucky when it doesn't

Takis_21:11:22

thank you for your time, hope it helps a bit on clojure.data.json

Takis_21:11:42

sometime i will re-test it if changes