Fork me on GitHub
#clojure
<
2022-10-18
>
Yehonathan Sharvit11:10:28

I’d like to create a custom version of TTL-based memoization where the cache is not invalidated in case the function throws an exception. 1. Is it possible to do so with clojure.core.memoize ? 2. Is there another library that provides this functionality? Remark: The function I want to memoize is a function that fetches data from an API that from time to time is unavailable.

p-himik11:10:17

Haven't checked myself but I'm 80% certain that this library either has that kind of functionality or has all the necessary building blocks to implement it on your own: https://github.com/clojure/core.cache

✔️ 2
skylize15:10:10

Stumbled on this while searching to learn what you mean by "TTL" (time to live). https://github.com/clojure/core.memoize/blob/master/docs/TTL.md

pppaul14:10:42

didn't know about core.memoize, been using core.cache all this time

Alex Miller (Clojure team)12:10:09

Clojure does not have any dependencies on other libs

❤️ 4
madis13:10:17

Hello. A #babashka related question. I'm trying to write a release script (for a ClojureScript library), that prepares & uploads JAR file to Clojars The rest of the scripts in the project are 100% babashka :star-struck: But I had some trouble getting some dependencies (e.g. slipset/deps-deploy) working under babashka. It's quite standard stuff (actually no Clojure compilation takes place, just CLJS files copied to JAR + POM generated), in other words straight https://clojure.github.io/tools.build/clojure.tools.build.api.html + https://github.com/slipset/deps-deploy Due to the dependencies issue, I started shelling out via (clojure ...) in babashka tasks, e.g.

release {:doc "Try to shell out to clojure"
         :task (clojure "-X:release :version 22.10.10 :library server/cljs-web3-next")}
> But this would need its own deps.edn , is slower and can't use the nice babashka helpers Now the question: 1. Do you have examples or recommendations of how to achieve it (prepare JAR & publish to Clojars) with just Babashka? 2. If not, any other recommendations how to make babashka + Clojure (JVM) work smoothly, reduce duplication (between bb.edn and deps.edn for example) etc. Thanks in advance, Madis

borkdude13:10:42

deps-deploy depends on some Java libraries which aren't available in bb. But I think clojars / maven also support REST apis that support releasing artifacts which you could use, if you want to do it all in bb - I've never tried

madis13:10:50

Great idea, thanks for suggesting! I'll see if I can publish over HTTP > From first glance looks like Clojars doesn't have (publicly documented) endpoint for uploading JAR-s > https://github.com/clojars/clojars-web/wiki/Data In case I go with publishing over HTTP, do you recommend babashka.curl or babashka/clj-http-lite or some other library for it? If I had to guess, one POST request should do it

borkdude13:10:58

It doesn't really matter which one you use I think. Both should work

madis13:10:44

Thanks borkdude, you're awesome. I hope your YouTube channel gets more views. Viewed bunch of your demos there!

borkdude13:10:25

Thanks :) Maybe I should do another video some time :)

emccue18:10:07

@U6P6QJTUZ if you find the api for clojars publishing can you doodle it down?

borkdude18:10:02

Perhaps someone in #C0H28NMAS can also help, but I also wanna know :)

emccue19:10:17

I never found what it was for maven central either ^

emccue19:10:44

i found that zip upload route but i don't get the sense that its really the intended path

rmxm15:10:31

hey, is there a way to force/shortcircuit cached/memoized functions returns? (consider having a function that is wrapped in ttl cache, is there a way for me to adjust the contents of this cache - like have a future running in parallel updating values)

p-himik16:10:41

> like have a future running in parallel updating values Why not just make that future call that function periodically?

rmxm16:10:22

could work, but... What if you are trying to avoid unwanted latency, so your cache is sort of: gimmie the best value for this at this time, doesnt have to be exact as it changes slightly. However during the cache window I want to run at least one update on the data. So for processing the cache is needed in order to provide latency. However the mechanics of the cache are not really required - well at least as long as there is backdoor to cached values.

p-himik16:10:11

Sounds like https://github.com/clojure/core.memoize should have what you need.

p-himik16:10:58

Specifically its memo-swap! function.

🙌 1
stopa17:10:19

Workers + Queues Hey team, How would you think about the following problem? I have a bunch of in-memory queues — potentially thousands of them:

app-a-q [a1, a2, a3]
app-b-q [b1, b2, b3]
...
And then I have a pool of workers:
worker-1, worker-2, worker-3
I want my workers to constantly take from the queues. But I want to ensure that all items in queues are handled serially. i.e if worker-1 is currently processing a1, then no worker can take a2. Buut they can b1. For queues right now I am using LinkedBlockingDeque. For workers I spin up a few (future (loop []…) blocks. Is there some async construct that can help me achieve my goal?

localshred17:10:56

async/alts! sounds nearly perfect for what you want, unsure if it will serially take from a if you had each worker using async/alts! on the same source channels/queues

❤️ 1
dpsutton17:10:57

each queue gets a go-loop taking from it and submitting to any number of executors would do it. You can balance the number of executors as you like and reports that it is done. The executors just get jobs submitted to them, they don’t care. And the many go loops just wait for the previous work to be done before adding the next elements in

❤️ 1
stopa21:10:50

Okay, I have a rough idea. Thanks team!

👍 1
jmckitrick17:10:13

I’ve been using multimethods for a while, but I’m trying dispatch functions that return a vector. I’m having issues getting [:default :default] to work, and combinations of :default as one of the terms. Calling the multimethod doesn’t seem to find a match other than a simple :default result, no vector at all. Any tips or tricks?

hiredman17:10:33

that isn't a thing

hiredman17:10:58

:default isn't some kind of wildcard in the dispatch

jmckitrick17:10:44

Hmm. Can I achieve that result, without the ‘implement this manually’ solution in the docs?

hiredman17:10:18

it depends, people often forget that multimethods dispatch using isa? not =, which means you can do stuff with dispatch hierarchies

hiredman17:10:26

like (isa? [String String] [Object Object]) is true

hiredman17:10:36

but basically no

jmckitrick17:10:08

I’ve done very little with the hierarchies, but I had my heart set on a simple decision matrix with a vector of 2 elements…

jmckitrick18:10:10

Which just led me to a better, simpler way. 👍:skin-tone-2:

jmckitrick18:10:58

:thinking_face:

jmckitrick18:10:39

I’ll check it out

Jérémie Salvucci20:10:59

Hi, I'm using docker to deploy a few applications. To use layers as efficiently as possible, I need to make reproducible builds. By this, I mean having the same jar archive at the end if no modification. Right now, I can't because zip archives include timestamp related information. Is there an option somewhere to get around this? I have seen a software to strip time related data but this seems like an adhoc solution…

kwladyka20:10:00

> because zip archives include timestamp related information. it can mean many things, show the code or cmd line output what you mean

hiredman20:10:45

clojure builds aren't really reproducible even without the timestamps

hiredman20:10:55

assuming you are aot compiling, there are a number of things that cause the compiler to generate slightly different bytecode every time it runs

hiredman20:10:31

I think the last thing I saw was the compiler has a map in it somewhere that uses object identity for keys, and then iterates over the map, so the order things come out of the map depends on System/identityHashCode, which is basically a pointer

hiredman20:10:07

there are people interested in this, I think also for things like nix packaging, but I am not sure if there is like a channel or something devoted to it

Jérémie Salvucci20:10:47

I see, at least with the last point, it's would be reproducible on my machine but I understand the issue here

hiredman20:10:44

System/identityHashCode is basically a pointer, so the order certain things are emitted in the bytecode will depend on where the memory allocator puts them

Jérémie Salvucci20:10:15

oh ok, if the address itself matters…

hiredman20:10:08

the thread I linked to above links to it, and apparently the person who opened that ask did some patching of clojure and claims to have reproducible builds

Jérémie Salvucci21:10:05

thank you, I'm digging into this!

kwladyka15:10:35

while we are in the topic here is my trick for cache to not repeat downloading dependencies after each commit:

# environment + clojure dependencies
FROM env as env-with-clj-deps
COPY deps.edn .
RUN clojure -A:uberjar:tests-deps:check-syntax-and-reflections:run-tests -Stree

# full image ready to compile and tests
FROM env-with-clj-deps as deps-with-code
COPY . .

kwladyka15:10:02

so the first step is COPY . . when the cache doesn’t match

kwladyka15:10:11

unless you changed deps.edn

Brice Ruth20:10:17

Good afternoon - I’m having a bit of a time doing a very simple Java -> Clojure call, based on [the example here](https://clojure.org/reference/java_interop#_calling_clojure_from_java) — it works if I use clojure.core/+ as in the example, but not if I use my own ns and function. Clojure code:

(ns example.oesa
  (:gen-class))

(defn hello
  "I don't do a whole lot."
  [x]
  (str "Hello, " x "!"))
Java (main) code:
package us.example.oesa;

import clojure.java.api.Clojure;
import clojure.lang.IFn;

/** Main class. */
public class Main {

  /**
   * Main entry point, prints `Hello World!`.
   *
   * @param args cli arguments (unused)
   */
  public static void main(String[] args) {
    IFn hello = Clojure.var("example.oesa", "hello");
    System.out.println(hello.invoke("World"));
  }
}
I started down this path using the Gradle + clojurephant plugin, along with the built-in [Gradle Application Plugin](https://docs.gradle.org/current/userguide/application_plugin.html) - but I’ve also tried to eliminate that as a factor and just try to run java directly from the CLI with the right -cp, but I get the same error as when I use my trusty gradle run java -cp build/classes/java/main:build/clojure/main:/Users/bruth/.gradle/caches/modules-2/files-2.1/org.clojure/clojure/1.11.1/2896bc72c90da8125026c0e61df0470a084f9ec3/clojure-1.11.1.jar:/Users/bruth/.gradle/caches/modules-2/files-2.1/org.clojure/spec.alpha/0.3.218/a7dad492f8d6cf657d82dcd6b31bda0899f1ac0e/spec.alpha-0.3.218.jar:/Users/bruth/.gradle/caches/modules-2/files-2.1/org.clojure/core.specs.alpha/0.2.62/a2a7ea21a695561924bc8506f3feb5d8c8f894d5/core.specs.alpha-0.2.62.jar us.example.oesa.Main
Exception in thread "main" java.lang.IllegalStateException: Attempting to call unbound fn: #'example.oesa/hello
        at clojure.lang.Var$Unbound.throwArity(Var.java:45)
        at clojure.lang.AFn.invoke(AFn.java:32)
        at clojure.lang.Var.invoke(Var.java:384)
        at us.example.oesa.Main.main(Main.java:16)

Brice Ruth20:10:23

I feel like I’m missing something very basic 😢

ghadi20:10:45

the infra (ns, vars, classes) created by your example.oesa namespace need to be loaded (unless you're doing something like AOT compiling your clojure, which I'm going to assume you're not doing if you are new)

ghadi20:10:01

so to trigger ns loading from Java, you need to call require:

ghadi20:10:55

IFn REQUIRE = Clojure.var("clojure.core", "require");
REQUIRE.invoke(Clojure.read("example.oesa"));  // this creates a clojure symbol from inside Java

ghadi20:10:07

then call your app stuff.

Brice Ruth20:10:45

ohhh … there is something in the clojurephant gradle docs that I added, related to AOT - but I just copy/pasted it, I’m not sure what it does or what else I need to do

Brice Ruth20:10:50

I’ll try this, thank you!

Brice Ruth20:10:40

that totally worked, I wish I had asked sooner

ghadi20:10:01

the snippet of java code I pasted is exactly equivalent to calling this from clojure: (require 'example.oesa)

👍 1
Brice Ruth20:10:11

I guess I didn’t understand what this part in the linked docs actually meant, 😂 > Functions in clojure.core are automatically loaded. Other namespaces can be loaded via require

waffletower23:10:12

Am learning something I did not expect from as->

(-> {:a 1}
    prn
    with-out-str)
"{:a 1}\n"
vs.
(as-> {:a 1} x
      (prn x)
      (with-out-str x))
{:a 1}
""

hiredman23:10:01

the docs for as-> do say it binds the name to the result, not that it replaces the name

hiredman23:10:24

so x is bound to nil after (prn x)

waffletower23:10:38

I am learning that yes

waffletower23:10:50

Years after using it

hifumi12323:10:14

When unsure on what threading macros will do, I always call macroexpand-1. I rarely use as-> so I sometimes have to pause and think about what's going on

waffletower23:10:31

I paused and thought about what was going on

waffletower23:10:58

Then I came here to chat about it

👍 1
🍵 2
hiredman23:10:35

it may depend on if you know the context about how as-> was added

hifumi12323:10:07

I've seen the swiss arrows library before, and I want to say as-> feels a lot like -<> from swiss arrows, but this is just my best guess on where it could've come from 😄

hiredman23:10:02

rich had some personal project somwhere that he ended up doing a lot of repeated let bindings in (let [a (f) a (g a) a (h a)] (i a))

hiredman23:10:32

as-> is that pattern as a macro

waffletower23:10:42

I played with swiss arrows right around 1.7 when I started with Clojure, and noticed as-> a bit later.

hiredman23:10:27

so if you read it as a let binding like that, what it does is pretty clear (and in fact that is how it macroexpands)

waffletower23:10:53

I use as-> infrequently when most of the forms need ->> but there is an outlier or two

seancorfield23:10:24

It's also worth mentioning that as-> is designed to be used inside a -> pipeline:

(-> expression
    (f)
    (as-> x (g 1 x 2))
    (h))

awesome 4
hiredman23:10:26

as-> was even originally named let->

waffletower00:10:31

Is there a known and perhaps well-discussed rationale for not having a corresponding as->> macro?

seancorfield00:10:40

What would that even mean @U0E2268BY? as-> uses a named symbol -- it's not "first" or "last" in anything.

hiredman01:10:54

Well, if as-> is only for use inside -> (I don't agree with that) then as->> would only be for use in ->>, would take the initial value last

hiredman01:10:06

(->> ... (as->> name ...))

seancorfield02:10:10

I didn't say "only for use inside" -- I only needed that part of its design is for that use case and that's why the arguments are expr, sym in that order.

waffletower02:10:59

The swiss-arrows -<>> isn't exactly what I had in mind, but is very promising for handling the situation I find myself in occasionally.. contrived example:

(let [h [:a :b :c :d :e :f :g]]
        (-<>> [1 2 3 4 5 6 7]
              (map inc)
              (zipmap h)
              (select-keys <> [:a :b :c])
              prn
              with-out-str))

waffletower02:10:09

"{:a 2, :b 3, :c 4}\n"

seancorfield02:10:37

And https://stuartsierra.com/2018/07/15/clojure-donts-thread-as which specifically talks about "Don’t use as-> by itself"

waffletower04:10:31

Oh not that cringe-worthy article again 😁

waffletower04:10:47

Stuart was a policeman in a former life

waffletower04:10:22

Sometimes there doesn't need to be a rule

p-himik04:10:24

Interesting how that article never mentions the context in which the macro appeared, one that hiredman mentioned. I wonder if Stuart knew it at the time of writing.