Was wondering, in Clojure, what's the best way to model an ordered list where I need to remove and insert from inside?

Normally, a linked list works really well for this. Since you can add elements in the middle easily, without needing to push out all the elements after or move up the ones before.

But the list in Clojure does not have such an insert/remove

Hard to say, I might just use a mutable LinkedList

Algorithms that require that sort of thing tend to be very imperative anyway

You could probably build a simple insert fn and remove fn for lists. I don’t come across many use cases that really need to operate on indices like that tho

Another option could be to use a sorted map

Again, depending on your use case

@didibus https://github.com/clojure/core.rrb-vector

Although, even a LinkedList isn't ideal if you are doing random accesses

@didibus also, alternatively, a sorted map?

sorted-map already suggested by @lilactown , didn’t see

I would probably use a sorted-map for something like that; alternatively I guess you can split and re-concat vectors via rrb-vector, but it’s somewhat more involved

If it’s a small enough of a collection, you can even just (remove …) (filterv …) from a vector?

(that would be linear time, but depends on the size of your vector might be OK)

Hum.. ya sorted-map wouldn't work, because I don't have anything to sort on. Though... I did think I could have an order key, and if I make it a double, in theory, I could take the element before and after in the current order, and make the new element the middle double value between the two

@didibus I’ve previously used rrb-vector where I split the vector and re-concat it…

subvec, catvec

rrb-vector is probably the best I guess. For my use case, I'll probably just go with a vector that I split, conj/pop and concat.

But I got curious

Was just showing someone how to build a quick todo list app, and got a bit choked on the drag and drop for moving items in the list around 😛

Mutable linked list could do as well I guess but :S

@didibus haha got it… Yea I would say sorted-map can work there, just have a map like {1 {:data 42}, 2 {:data 43}}

Sort by the key…

Re-arranging things can get complicated though…

Hum, that's kind of what I had, but if you have: {1 {:data 42}, 2 {:data 43}, 3 {:data 34, 4 {:data 34}} and you want to move :data 34 after :data 42 ?

rrb-vector it is I think 🙂

You need to kind of iterate over all, increment, and all that

Yea, I agree, not the best… rrb-vector 💪

Which is where I thought, could make the key a double, so I would just turn: 4 {:data 34} into {1.5 {:data 34} haha

@didibus you could 🙂

It just started to feel like I was stretching things a little far with that 😛 Thought there has to be something more obvious

yea… rrb-vector is my vote for sure… more idiomatic; less accidental complexity data around

Which I guess there isn't, this is actually quite a non trivial thing if you want to be immutable and try to approach O(1)

Ya, I heard some rumors that rrb-vectors has a lot of known bugs in its imlementation?

Was that old/fake news ?

Somebody more familiar would have to chime-in… I am not up-to-speed on it; I’ve used it but it has been years, it wasn’t recently

It’s moved under clojure / in github… it would hope it’s relatively OK to use

Ya, I'll give it a roll one day

thx all

rrb-vector does have at least one remaining bug that I know about, but as far as I know it requires fairly large vectors to cause it to occur. I wouldn't bet millions of dollars on that guess.

I would think that for a to-do list, you aren't necessarily needing nanosecond-level response to changes from a UI in the order, and the lists aren't thousands of items long, so perhaps a Clojure vector/list, even with linear time required to arbitrarily insert elements at internal positions, might not actually be noticeable performance-wise?

Performance question.

(defn -merge
  ([xsa] xsa)
  ([xsa xsb]
   (concat
    (butlast xsa)
    (list (concat
           (last xsa)
           (first xsb)))
    (rest xsb))))

How are you measuring? My guess is it is taking that long to print the result

Using tufte to profile. Wrapping the function in a count so nothing gets printed.

Last is always going to kill, you could try swapping the inner concat for lazy-cat

If you write your own lazy-seq thing instead of using concat and friends like that you can take advantage of the struct of lazy seqs to do much better

Hmm. Instead of a butlast followed by a last on the same thing, I wonder if a (split-at (dec (count xsa)) xsa) would be faster. Kill two birds with one stone. I'll try out the lazy-cat too.

(fn f [a b]
  (lazy-seq
   (let [f (first a)
         r (rest a)]
     (if (seq r)
       (cons (first a) (f (rest a) b))
       (cons (concat f (first b))
             (rest b))))))

using count with split-at as the same problem as using butlast and last

you have to traverse the seq multiple times

so you end up both not being lazy, and doing multiple traversals, where want you want to do a lazy traversal is such a way that those lazy traversals can be stacked, so no work is done until you force the final product, which does all the work and only does a single traversal

but there is a good chance your other code elsewhere is forcing seqs, so who knows

In the absence of hitting bugs in the rrb-vector library, that is the kind of thing it is made to be fast at, i.e. lots of split and concat. operations.

When a function looks like this, how does it work? For example; it seems like there are two bodies. Does it pattern match on one of them?

(defn env->UserCredentials
  "Create a UserCredentials instance from environment variables, defaulting to
  CONSUMER_KEY, CONSUMER_SECRET, ACCESS_TOKEN, and ACCESS_TOKEN_SECRET"
  ([]
   (env->UserCredentials {:consumer-key      "CONSUMER_KEY"
                          :consumer-secret   "CONSUMER_SECRET"
                          :user-token        "ACCESS_TOKEN"
                          :user-token-secret "ACCESS_TOKEN_SECRET"}))
  ([env-mapping]
   (let [credentials (map-values #(System/getenv %) env-mapping)]
     (assert (every? some? (vals credentials))
             (str "Failed to read credentials from env; one of "
                  (vals env-mapping) " could not be found."))
     (map->UserCredentials credentials))))

it will choose a body based on the number of arguments passed in (the function’s arity)

Awesome and why does this function name have a -> in it? Is it some kind of convention for something?

yes, just convention

I think it’s a convention for “to” or “transforms into”

so “env to User Credentials”

Ah cool; that makes sense.

Thank you! 🙂

@tiotolstoy the multi-arity is a common way for providing defaults (like in the example above); you could even have completely different behavior, but that’s less common (maybe even weird and not recommended)

Yep; personally in love with pattern matching on function arity. Just wanted to confirm that’s what I was looking at! @raspasov

Why can you not have many expressions inside the function body of main?

(defn -main
  [& arg]
  (for [user_id following_ids]
    (mute_user user_id)))

If I included (def user_id ….) above for it would error out with some error about inline-def

I'd suggest taking those kind of questions to #beginners

Def in clojure always creates a top level global definition, not a definition in whatever scope you are in, so it almost always a mistake to nest it in some other scope

OK sorry hiredman. I’ll use beginners next time 🙂

@tiotolstoy The main reason for that is that in #beginners folks either are beginners or have opted-in to helping beginners so the patience and tolerance is a lot higher there. The expectation for #clojure is that folks have a solid knowledge of the language and tend to be asking more esoteric questions that are likely to have opinion-based answers...

Understood Sean 🙂

I’m running into a weird macro-related problem. Here’s a minimal example to reproduce:

(def data (constantly true))

(defmacro def-fn [] `(defn ~'foo [] ~data))

(def-fn) ; throws java.lang.IllegalArgumentException: No matching ctor found for class clojure.core$constantly$fn__5672

how popular is memoization in clojure? I am looking at a case: There is a service that returns some configuration details, the service is http and stateless. I see in the code they call the service during bootstrap, repack the values into more usable structure and keep that in atom. I was thinking this should be memoized, but then I still have a bunch operations on raw data happening everytime someone wants a particular piece of config, so I would have to memoize all helper functions that access the config?

i'm a relative macro noob, but i'm trying to write some wrapper macros around spec and finding it... challenging. specifically, i'm trying to dynamically generate the keys that i want to define specs for, and i'm being confronted with s/def itself being a macro, and not capable of understanding variables example, this works:

(defmacro def-spec-test1 [k]
  `(s/def ~k string?))

(defmacro def-spec-test2 [k]
  `(let [k# ~k]
     (s/def k# ~string?)))

(let* [k__29626__auto__ :foo/bar] (clojure.spec.alpha/def k__29626__auto__ #function[clojure.core/string?--5410]))

Is anybody here using any of the ‘complementary’ util libraries, such as Specter, Medley, or Tupelo? If so, how’s it working out for you?

Does anyone here have experience with DynamoDB and https://github.com/cognitect-labs/aws-api ? I am using the Scan operation, but I get Items in the response in the DynamoDB format, was wondering if there was a simple way of unmarshalling like in the JavaScript sdk?

Amazon provides augmented SDKs for certain services (some examples are KMS, DynamoDB, Kinesis). Those service-specific libraries handle service-specific concerns (e.g. key-stretching and caching in KMS, consumer state management in Kinesis)

aws-api does not attempt to handle things that are not written down in Amazon's API service descriptors. The DynamoDB storage payload format is not in those descriptors.

you can make a helper function that converts a clojure map {:foo "bar"} into {"foo" {"S" "bar"}}, which Dynamo payloads require and vice versa

AFAIK, this is the only special handling that DynamoDB requires (besides respecting rate-limits, which applies to all AWS apis, but moreso to DDB) @thedanpetrov Happy to continue discussion in #aws

A History of Clojure by Rich Hickey https://clojure.org/about/history

Is it possible to use derive for multimethods when the accessor is a string? e.g. (derive "square" "rect") Or if not, is there an efficient way to coerce the string accessor to a keyword? e.g. (get-area {:shape "square", :w 10, :h 10}) and have that use the (defmethod get-area :rect (fn [{:keys[ w h] (* w h ) }

What's a nice way to def with metadata? I want to re-export some functions from a different namespace.

you can specify a metadata map as an arg to intern, you could literally apply the metadata of the original to the re-export on definition

user=> (intern *ns* (with-meta 'j (meta #'clojure.string/join))  clojure.string/join)
#'user/j
(ins)user=> (doc j)
-------------------------
user/j
([coll] [separator coll])
  Returns a string of all elements in coll, as returned by (seq
coll),
   separated by an optional separator.
nil

but re-exporting has many gotchas, and it's often more straightforward to design your namespaces differently rather than having a facade

what about this? (haven't tried it) https://github.com/ztellman/potemkin#import-vars

@isak Thanks! That seems to do exactly what I was asking for 💯

@noisesmith > it's often more straightforward to design your namespaces differently rather than having a facade Can I dig deeper in this line of thinking somewhere?

this discussion covers it well I think, with @tbaldridge representing the viewpoint I'd advocate https://groups.google.com/forum/#!msg/clojure/Ng_GXYSrKmI/q0cxahB3BAAJ

you will likely recognize a few names in that discussion

also my example with intern above works around the complaints in the thread about go to source finding the wrong location, and transitive dependency bugs

(though I don't even recommend that - I just think it's better than the magic import-vars attempts)

TLDR: learn to love 10K line files, it's fine

that's not what is advocated in that thread by Timothy at all

You don't see how the consequence would be larger namespaces? (I don't have a problem with it)

> Anything not in the API that should be unpublished to users is in other namespaces that are imported and wrapped by vars in the main namespace. the difference is that it's done by making a new definition defining the API, not re-exporting the var from another ns

the idea of having a layer that doesn't expose details that you don't want people relying on is fine (it's a fundamental principle of structured programming) - the difference is a tactical one, the mechanism for doing that layer

Ok, fair enough

Is there a more reasonable way to add a self-referencing multi-arity multimethod than to either: - Create a separate multi-arity function and then use that function, e.g. (defmethod f :x ([_ a] (other-f a)) ([_ a b] (other-f a b))) - Call (.addMethod multifn ...) manually ?

@seancorfield hi, I am trying to use your depstar library this work

clojure -A:depstar -m hf.depstar.uberjar app.jar
java -cp app.jar clojure.main -m api.core

clojure -Spom
clojure -A:depstar -m hf.depstar.uberjar app.jar -v -C -m api.core
java -jar app.jar
Error: Could not find or load main class api.core
Caused by: java.lang.ClassNotFoundException: api.core

while I am thinking about paying model dependent on start app time and processing time AOT start to be more interesting. I can build app even with Java 14 or 11

the java version shouldn't make a difference - clojure doesn't use javac to emit bytecode

well, in theory it could make a difference because the compiler uses reflection to resolve methods

and sometimes you get subtile behavior changes across jvm version like arrays of methods being returned in different orders, etc

and new java features actually let you compile totally different classfiles for different java versions

@kwladyka You're missing (:gen-class) in the ns of api.core.

Oh I don’t know why I thought this is not needed anymore with deps.edn and depstar

You're the second or third person to trip over this in the last week so I'm going to update the depstar README to clarify that.

If you create a project with clj -A:new app api.core then clj-new adds (:gen-class) for you, just like lein new and boot new do for app projects.

now it works 👍

thank you

(if you use clj-new to create your app project, your deps.edn will already have an :uberjar alias to save you some typing)

Do you recommend to use AOT in Dokcerfile so when I am sure java version is the same? I am not an expert in this topic. totally. It can gives me better performance maybe, so it sounds interesting. I think I didn’t try AOT before, so I have no opinion.

the only "performance" difference is you might get faster start times

I want to use this with google run / functions, so this matter

because it is real cost

Whole-application AOT in order to improve startup time is a valid use.

ok, I have impression people try to avoid AOT

oh I will copy this solution with :uberjar in deps.edn

AOT is preferred on the operations side of things, especially with serverless and containerized applications in the JVM.

aot is gross

good, this is what I am planning to do

because you are generating and persisting bytecode to disk it can cause issues with clojure's more dynamic features. you can usual avoid those issues if do things like only aot for deployed artifacts and not locally while testing and running repls

but then sometimes things will load fine in clojure, but break when you aot them, so then sometimes you check things in because it looks good and works locally, and then go to build and boom

While I hear a lot of this things I don’t know any specific case which really happen to somebody

that said, these days you see a lot fewer people struggling with aot related issues

that's what good CI/CD pipelines are for. i agree that it can impede local development.

if this is blow up during compile to uberjar, then this should be fine

ah in the context of PDF generation… wkhtmltopdf generate much different PDF (everything looks so bad) in docker, than in my local environment. Even while this is the same version of wkthmltopdf. I think I will try this alternative solutions faster, than I thought 🙂

what distro? i've hit this before in node. it's tricky to get all the right fonts and styles in place.

I am trying this with FROM clojure:openjdk-14-tools-deps-1.10.1.502-buster as deps

this is much more broken, than only fonts. wrong sizes, broken pages, wrong width of content

I am going sleep, it is too late. Thank you all for your help. Good night!