This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-10-21
Channels
- # announcements (13)
- # babashka (29)
- # beginners (52)
- # calva (95)
- # cider (18)
- # clj-commons (7)
- # cljs-dev (42)
- # clojure (121)
- # clojure-australia (1)
- # clojure-dev (39)
- # clojure-europe (36)
- # clojure-france (4)
- # clojure-greece (1)
- # clojure-italy (20)
- # clojure-nl (3)
- # clojure-portugal (1)
- # clojure-uk (7)
- # clojurescript (47)
- # conjure (2)
- # cursive (9)
- # datalevin (5)
- # datascript (8)
- # datomic (68)
- # defnpodcast (2)
- # deps-new (5)
- # fulcro (18)
- # graalvm (21)
- # gratitude (9)
- # jobs (6)
- # jobs-discuss (17)
- # leiningen (3)
- # lsp (80)
- # lumo (1)
- # malli (9)
- # mount (2)
- # off-topic (16)
- # other-languages (8)
- # podcasts-discuss (19)
- # reitit (5)
- # remote-jobs (5)
- # shadow-cljs (29)
- # sql (5)
- # tools-deps (13)
- # vim (11)
- # xtdb (19)
Hi everyone. I have a question on macros: I have a predefined list of items. I have a macro that takes a variadic argument. Is there any way to spread my predefined list into the macro? e.g.
=> (defmacro v [& args] `(vector ~@args))
=> (v 1 2 3)
[1 2 3]
=> (def a '(4 5 6))
The value of a
is constant, and def
’ed, as above.
Is there any way to use the macro v
with the contents of a
?as long as a
is not a local binding, you could do something like:
(defmacro vapply [macro xs]
`(~macro ~@(eval xs)))
> (vapply v a)
[4 5 6]
(let [b [1 2 3]]
(vapply v b))
;; Unexpected error (UnsupportedOperationException) macroexpanding vapply
;; Can't eval locals
another alternative is (eval (apply list 'v a ))
Depending you may have to replace those nils with actually values but I forget the order of &form and &env, env you can pass through, form http://will.be goofy
applying macros isn't the answer, the answer is stepping back and avoiding whatever it is that is making you want to apply macros
Well, I have a macro that I don’t control, and I have a seq of predefined data that I want to give to it, and I want to use that seq elsewhere in the code as well. Until now, that very large seq is copy/pasted between the 2 areas. I really want to let the macro use it
If I had control of the macro, it would be very easy. Just redefine it to use a seq. It actually turns the arguments into a seq, so it’s trivial.
At that point, I could write a macro just like the one that current exists (taking a variadic argument), collect the args, and pass them into my new, better-designed macro.
I would double check the library docs or whatever to make sure that is actually the case
It’s a DSL. The idea is to call it with something like:
(defprog [a b c]
operation1
operation2
(op3 a)
(op4 b c))
I’m trying to get fancy and construct programs to run in itMy experience with that sort of thing is largely around core.logic, which ends up having function versions of the operations you need for that sort of thing, just not well documented
Just stumbled upon an old Clojure mailing list thread: https://groups.google.com/g/clojure/c/o43RyH_ua8I
(defmacro let-map
"Equivalent of (let [a 5 b (+ a 5)] {:a a :b b})."
[kvs]
(let [keys (keys (apply hash-map kvs))
keyword-symbols (mapcat #(vector (keyword (str %)) %) keys)]
`(let [~@kvs]
(hash-map ~@keyword-symbols))))
user=> (let-map [a 5 b (inc a) c [a b] d {:foo c :bar (* a b)}])
{:a 5, :c [5 6], :b 6, :d {:foo [5 6], :bar 30}}
That's awesome!@didibus I have this one in a work project:
(defmacro ->map
"(->map a b) ;;=> {:a a :b b}"
[& symbols]
(assert (every? symbol? symbols))
`(hash-map
~@(interleave (map keyword symbols) symbols)))
Hi. Is there any framework (java also considered) that supports this functionality: 1. one master and multiple slave instances 2. the master sends a task to a queue, and only one slave instance will fetch task from the queue, process it 3. the master waits and read the result of that task
@UGC0NEP4Y I’m the author of Proletarian (https://github.com/msolli/proletarian). It’s a Postgres-backed job queuing and worker system for Clojure. Please share your use case in #proletarian, and let’s see if it can be of use to you.
FWIW I was actually considering proletarian at some point, but decided against it given that I already have a listen/notify implementation of my own.
> FWIW I was actually considering proletarian at some point, but decided against it given that I already have a listen/notify implementation of my own. Thinking on this, I feel Redis as a middleware might achieve this too.
Redis is just a transport. It's neither framework not a library. It's definitely possible to use it to achieve what you want, but it would also be possible with many other transports.
@UTF99QP7V Re. listen/notify: No plans yet. Polling works great, it’s simple and performant. I’d love to look into it if there’s interest and valid use cases for it.
@U2FRKM4TW I’m interested to know more about your implementation. Do you have anything to share?
Fair enough. Would you mind if I pinged you at some time in the future to pick your brain about this if/when I venture into implementing listen/notify in Proletarian?
I doubt I would be the right person but sure, I don't mind. :) On the topic of polling vs listen/notify - if I'm reading the code of Proletarian correctly, you're essentially busy-looping with the polling. There are no configurable delays of any kind. It's a problem in an of itself - apart from an unnecessary load (albeit, probably a low one), it will pollute PostreSQL logs if you have configured it to log queries.
No, it’s not quite how it works. While there are jobs to be done, yes, it’s going to loop around and get the next item. But when there are no jobs, the Queue Worker (backed by a ScheduledExecutorService
) is going to wait the configured amount of milliseconds (`:proletarian/polling-interval-ms`) before polling.
Ah, right, I see now, thanks for the explanation. Although it will still pollute the logs.
Sure. Though I wouldn’t run Postgres with log_statement = 'all'
i production, so the query statements wouldn’t be logged at all.
Has anybody done anything with pulling from OSX Keychain in Clojure(Script)? Thinking about how to have secrets in local dev config but not stored as plain text, or plain text in environment variables, fetched using something like an edn reader macro… e.g.
{:username 'my-user'
:password #keychain 'secret-name'
...
}
Not OSX Keychain but maybe https://github.com/elasticdog/transcrypt ?
i've always treated dev credentials as not super secret. The dev setup should be recreateable and throw awayable if necessary
mostly just repl env stuff, when we are trying to hit dev environment when not using fakes
so was looking at calling out to shell with something like:
(-> (sh "security" "find-generic-password" "-s" "AD Creds" "-w") :out)
but was checking if any recommendations… 😉Probably dragons indeed. As dpsutton mentioned, usually treating dev credentials (assuming you don't have your dev setup available from the outside world) as something not that secret is fine, and using plain text files with dev credentials is an approach I've seen incomparably more times than any other.
sadly they are dev credentials, and not local credentials at this point… plus it is Active Directory credentials on top of that
Seems like there's a slight mixup of terms. At least in my experience "dev" in "dev credentials" means "used during development". Which usually implies that a single developer has everything that's needed to work on the software in question on their PC, thus automatically making dev credentials to also be local ones. But given that you use AD + Vault for that, I think in your case "dev" means something different or that implication of a single developer having all the needed pieces locally does not hold. Either way, that's just a rambling attempt at clarification. Regardless of the specifics, I'm absolutely sure that shelling out is the simplest and the easiest way to go here.
yeah, sorry… they are the Deployed Shared Development Environment credentials, not local dev credentials… fell into the terms used in the job, which I do think are goofy and confusing
Is it possible to extend a protocol to another protocol, i.e. say that everything that satisfies LowerLevelProtocol also satisfies HigherLevelProtocol (and provide its implementation in terms of the lower-level protocol functions)? I guess not?
directly, no
there are a couple of techniques though
one path people sometimes use is to extend using the generated interface, probably the most common, but I'm not in favor of it because you will miss extensions via metadata etc
another is to have a default extension in Object that determines whether the concrete type you have supports the protocol and then installs the extension dynamically (to the concrete type), after which it can route directly
Could you be so kind and explain that a little more? 🙏
Found it, Extending Protocols to Protocols at p.22 Thanks!
and another very advanced technique is to directly manipulate the internal protocol map
another way is if the protocol isn't used directly but rather there's a function that itself dispatches to the protocol you can do whatever you want before invoking the protocol. Requires more of a design before you begin though
we've taken to treating this as a best practice and you can see this prevalent in spec for example
Here's the example: https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L44-L50
And then here: https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L167-L184
You can see that there are functions defined with the same name but without the *
at the end, and they just call the protocol methods.
in spec 2, we're using those outer functions to do more special handling
Thanks, still trying to wrap my head around how this technique is a way of extending ProtocolB from ProtocolA (the original question, if I understood it correctly). I’ve faced this problem before and what I’ve wanted is a way to use all the methods in say ProtocolA in another ProtocolB with B having some more methods.
if (satisfies? ProtocolA x) => true
then (satisfies? ProtocolB x) => true
(I understand this is stepping into OOPs territory)it relies on impl details, but Clojure leaves those details accessible for those that understand the tradeoffs
we've taken to treating this as a best practice and you can see this prevalent in spec for example
Like, if you have protocols A and B, and you want all Bs to support A's methods, you can always have
(defprotocol A
(do-thing []))
(defprotocol B
(do-other [])
(as-a []))
And then your B protocol requires that implementors have some way to produce an implementation of AIt has the pro of encoding the relationship between the protocols explicitly, with the con of requiring explicit conversions by convention (that might be missed since the basic code might work regardless)
thank you!
Not to derail, but does anyone have any experience using ImageIO, maybe with twelvemonkeys, in production?
Yes, we use this combination for all our image manipulation.
We use it to auto-correct rotation when a member uploads their photos. We also do cropping and resizing with it.
(for privacy reasons, we strip EXIF after initial processing)
Also I think you must have ran into this exact issue - will imageio be enough/have a way to interpret and later strip the exif data?
Yes to the latter Q -- that's exactly what we do. As to a separate server, no, we do a bunch of ImageIO/TwelveMonkeys stuff inline in a couple of our processes. We do have a separate "photo queue" process that does more of the heavy lifting, but the reason for that is that it also uses external AI processes to analyze images and so we have to throttle the calls due to rate limiting thresholds on those services.
Oh, on the EXIF stuff, we recently added com.drewnoakes/metadata-extractor {:mvn/version "2.16.0"}
since ImageIO didn't give us everything or didn't work for all image types. I'd have to dig in JIRA to see exactly what the root issue was there.
I believe it is @U07QKGF9P who mentions somewhere in https://clojurescriptpodcast.com/ that image processing should run in its own VM because there is no way to make the processing C libraries bug free and thus you open yourself to be hacked by a well-crafted image. So I would run anything like ffmpeg in a lambda. ImageIO is Java so it is likely far safer than C but I am not sure it is 100% safe.
We have some code that relies on ffmpeg for doing image resizing and it's missing the rotation metadata on some jpegs, which causes wierd rotated preview images
I'm trying to figure out a bug with a cli4clj application, which I understand is basically a wrapper around clojure.main/repl
. I have an input that looks number-like (ex. 1234/abc
), but it uses clojure's read
and that interprets the input as an invalid number. Is there a simple way to hack the reader to treat inputs as strings instead?
That works just fine, and I have other workarounds too, but I wanted to know if there was a simple approach I could take to avoid that
not with the clojure reader - namespaces can't start with numbers in symbol literals (though the symbol
function will happily turn any string into a symbol)
if the reader interpreted input as strings, it would just return the input, and strings eval to themselves to that repl would be identical to identity
right?
Well, in a cli application you'd hand those strings off to the eval function to be processed anyways, and I don't really care about data types there, I just need strings
i don't follow. if you eval the string "1234/abc" you're gonna get either the exact same string back or the same error as above
I have a cli prompt that goes something like this:
# mycli > arg1 my-id/here
# (output)
# mycli > arg2 1234/abc
# (<crashes>)
I'm just using the inputs as strings to figure out which commands to run, all the inputs are treated as strings anyways. So I don't actually want "reading" to happen. I've got my own custom eval - that's what the cli4clj framework supplies.The "read" unfortunately crashes on valid program inputs because not all valid program inputs are valid edn. I'll file an issue to allow plugging in your own reader.
i'd question why you are using the reader at all here. Why are you using clojure.main/repl ?
I'm just investigating a bug report on a legacy tool we have that crashes on unusual inputs. This application was made with cli4clj, which wraps clojure.main/repl.
Hey, does Clojure community has any preferences regarding the best approach to sessions for web? I usually went with a stored session + session-token + cookie, and now looking towards a JWT stored in the same cookie If this was discussed already – a pointer would be enough, thanks!
Can't speak for the whole community but it has been argued time and time again that JWTs are largely misused, especially when it comes to scenarios that can be solved with the solution you're already using. I haven't seen the article above (and thanks for the link, I'll check it out), but here's another one, JWT-specific: http://cryto.net/~joepie91/blog/2016/06/13/stop-using-jwt-for-sessions/
Thanks @U050ECB92! Thanks @U2FRKM4TW!
@U2FRKM4TW not following the critique there. I think JWT that contains only a {user-id: 1}
and is stored in a secure cookie is alright, isn’t it?
But yeah if we want an individual token invalidation we’ll have to store them somewhere.
UPD:
Ok, I got it, yeah, I’m reinventing the session.
Thanks!
Now reading the second part