if you're using Lein, try adding your log4j.properties to the project resources folder and update :dev profile in project.clj file `:profiles {:dev {:resource-paths ["resources"]} :uberjar {:aot :all}}`

It can save significant amounts of memory for large vectors. For example, a vector of long values takes about 1/3 the memory using vector-of vs. vector

I don't have any benchmarks to show

vector-of should only use slightly more memory than corresponding Java primitive array, e.g. probably 5 to 10% more.

So, the primary purpose is space efficiency?

I am thinking of adding a note here: http://clojuredocs.org/clojure.core/vector-of

Ya, it turned out vector-of doesn't allow primitive math over it. But it has faster inserts

If I remember correctly

So basically, internally, it stores the elements unboxed, but everytime you put or take things from it, they'll get boxed

Got it. These are great points!

I suspect that because it is more space efficient, there may also be some operations that it is faster at than vector, but that is a guess not checked by any benchmark measurements I have done.

For the prime example, a really easy way would be to define the sieve recursively, and simply memoize To get the function var bound to the memoized version I think you could either do a def instead of defn on the anonymous form e.g. (memoize (fn [] ...)) , or potentially just rebind the var again with def outside

I also want to retain the performance benefits of a sieve. Wouldn't this be slow?

If you bind the initial "function" var to the memoized helper function, you'll get the benefit of memoization (constant-time return of previously calculated results) and automatic calculation AND storage of new results. defrecord and deftype aren't even required.

Considering the nature of a prime sieve is entirely numeric, there's no need for any types, complex or otherwise, beyond those available in core Clojure.

I doubt memoize will be equally fast though. Here is an exploration of the fastest possible sieve implementation in Clojure which relies on a boolean-array. I reckon memoize is associative? That's quite an overhead for hot loops like this. Thanks for chiming in though, good to hear your thoughts.

https://clojureverse.org/t/eratosthenes-party-time-a-k-a-feedback-wanted-on-this-implementation-of-eratosthenes-sieve/3801/16

Ah, so that code emits a lazyseq, which would elide unnecessary calculation for for anything not accessed. So if you want to know if the sieve contains x, you can either do a some (linear) or contains? (constant or log) on the returned seq. True, because this sieve impl returns the entire set as a lazyseq at once, you won't gain much from memoize. But unless you are searching for a low prime, you pay a very similar memory overhead for what is essentially the same constant-time lookup as an indexed memoize algorithm.

Not quite getting your point. Maybe I am missing something. Are you talking about the concat? Sure that's a lazy seq, but it is backed by the boolean array. So the performance gain of not realizing the seq should be negligible?

True, the data behind the lazy seq already seems to be realized, so... it's computationally and memory-wise comparable to a memoized indexed solution.

... as long as any x you have is (< 3 x n). For any x higher than a previously calculated n, an indexed/memoized solution can be engineered to pick up from the previous n, where this solution would regenerate an entirely new seq.

Fastest performance and immutability don't go hand in hand

But you almost never need a deftype

Generally, you'd use an array, or a Java mutable collection

Transients can also help sometimes

As a compromise between the two

In sieves example, if you wanted to reuse the generated prime->boolean cache between invocations, I would just put it in a def. That's all

(def cache ...)
(defn sieve [n]
  ...
  cache 
  ...)

If you are OCD about encapsulation. You can have your function close over it instead:

(def sieve
  (let [cache ...]
    (fn [n] ... cache ...)))

But, once you're back to the mutable world, multi-threaded use of shared variables become a problem again. So keep that in mind. If you fill the cache when its initialized. And make the sieve fn only read from it its fine. But if say two sieve fn are called concurrently, with the same prime, and they both try to grow the cache for it at the same time, you might get weird race conditions.

Another way to handle this is to have the sieve fn take and return the cache.

(defn sieve 
  ;; returns a cache up to n 
  ([n] ...)
  ;; returns true/false based on cache
  ([cache n] ...))

Yes, I have OCD about encapsulation. def with let seems nice. So is the idea of taking and returning a cache. Will try these out and bench them as well. Thanks.

In fact the with and without cache idea is wonderful! You have shown me a new way of thinking about things, and this seems more functional as well.

The problem with keeping a namespace global cache is that now you can have only one cache. For sieve, I guess that's okay, but not okay generally IMO. Also, the cache should not be the implementer's headache.

Looks like mailgun uses a third-party service for the actual sending. If you have a mailgun account already, this would be a strong choice. postal seems to send the message via SMTP(+TLS) directly. It runs a higher risk of mails being classified as spam unless your domain DNS is configured correctly. It also requires java 1.8; none more recent.

@UGPM7GTKJ https://github.com/drewr/postal#encryption-gmail-example Does this remove the concern with spam classification? I'd be delegating email sending to google?

No, the DNS measures I meant are unrelated to the encryption in transit. This might help: https://en.wikipedia.org/wiki/Sender_Policy_Framework

You can use postal with any email provider that supports smtp so it might be a better choice if you want to have some flexibility in switching between them.

@UGPM7GTKJ thanks for the link and info @U06BE1L6T not being classified as spam is more important. And i understood correctly i need mail server to run postal? https://github.com/drewr/postal#dependencies

@U45SLGVHV iiuc, postal speaks SMTP directly to the recipients' MX, so no, you shouldn't need to run a mail server yourself. But if the originating ip/host of the email doesn't match the SPF record, you stand a slightly higher risk of your mail being categorized as spam by the recipient.

if there's a designated domain server you can use as a relay for outgoing emails, you can put that concern to rest.

Right, you can use something like AWS SES or tons of other email services as well as your own email server (if you want) if your app uses plain SMTP. You can basically defer that decision and change it later too.

E.g. at work, we have a standalone application which customers install on their own servers (on-prem) and the app uses postal to send scheduled reports. One company uses their own email server to send those reports while I use SES for testing.

So you want x to start at zero, used a (< x n) when condition, and increment x instead of decrementing it, is that right?

Precisely, yes.

oh

I should just do [x 0] and (< x n) lol

Right :-)

Ok, that worked. Thanks 😄

There is also https://clojuredocs.org/clojure.core/dotimes just as an aside for cases like this where you don’t need to manage iteration cases.

Which is just a macro which expands into just what you are doing 🙂 https://github.com/clojure/clojure/blob/master/src/clj/clojure/core.clj#L3326-L3329

👍 nice, thank you 🙂

Turns out I have to use something else anyway because the testcase expects a string and not printed lines.

Probably a for loop then if I recall correctly since it can collect result?

That sounds very reasonable to me!

Ok, managed to figure it out

(defn draw-stairs [n]
  (clojure.string/join "\n"
    (for [x (range 0 n)]
      (str (clojure.string/join (repeat x " ")) "I"))))

there are dynamic vars that pprint uses that help control that behavior (though even that can be awkward)

user=> (apropos #"^\*.*print.*\*$")
(clojure.core/*print-dup* clojure.core/*print-length* clojure.core/*print-level* clojure.core/*print-meta* clojure.core/*print-namespace-maps* clojure.core/*print-readably* clojure.pprint/*print-base* clojure.pprint/*print-miser-width* clojure.pprint/*print-pprint-dispatch* clojure.pprint/*print-pretty* clojure.pprint/*print-radix* clojure.pprint/*print-right-margin* clojure.pprint/*print-suppress-namespaces*)

it's ... a start

user=> (binding [clojure.pprint/*print-miser-width* 2 clojure.pprint/*print-right-margin* 4] (pprint {:a 1 :b {:c "asdsada" :d "zzzz"} :z 12312}))
{:a
 1,
 :b
 {:c
  "asdsada",
  :d
  "zzzz"},
 :z
 12312}
nil

this isn't perfect, but it does work precisely almost as you want for your input

ser=> (binding [clojure.pprint/*print-miser-width* 15 clojure.pprint/*print-right-margin* 18] (pprint {:a 1 :b {:c "asdsada" :d "zzzz"} :z 12312}))
{:a 1,
 :b
 {:c "asdsada",
  :d "zzzz"},
 :z 12312}
nil

Thanks!

sounds like reduce is what you are looking for

(reduce 
  (fn [acc f] (f acc))
  x
  (repeate n f))

That's what I felt too, but couldn't figure out how! Repeating the function rather than the object is the key here. Thanks! 🙂

first class functions are always your friend)

In python and R too, but the amount of problems you can solve with reduce alone is quite new to me 😄

what about repeatedly? (repeatedly #(frob x))

where repeatedly also takes an N arg (repeatedly N #(frob x))

user=> (repeatedly 10 rand)
(0.8080800356339141 0.8469572918844156 0.6770624642868489 0.9461635939606167 0.5870982429331622 0.28680625632485723 0.1834445338833387 0.24913157800173336 0.5788381193091194 0.08541291908430448)

Wouldn't work in case x is immutable, as x in #(frob x) wouldn't get updated

you said in the start that f mutated x

but yes, if you need to use return values and x is immutable, use reduce

or iterate / take n

user=> (take 10 (iterate inc 0))
(0 1 2 3 4 5 6 7 8 9)

Sorry if it wasn't clear, but the problem is indeed that the two approaches I thought of first either didn't work for immutable objects (repeatedly) or not for mutable ones (iterate). The reduce option seems to be good for both 🙂 (except that in case of immutable, my sequence will be filled with identical objects ofc)

which sequence? the reduce version just gives you the last return value

Ah yes, ofc, sorry, was confused. Got used to using reduce in combination with adding an element to a sequence 🙂

CLJ-2555 is based on Rich's ideas coming out of CLJ-1906 and is headed towards 1.11

Neat

there's an async variant too for core.async, not sure if Ghadi has posted that yet

Either or

use the Float version

But just know that they don't do the same thing

not necessarily

So it depends what you're trying to do

(edn/read-string "7") will be different from the Float/parseFloat version

Edn read-string will parse following the Clojure literal parse rules

While the Java Number/parseNum methods have their own rules

(edn/read-string …) is better because it automatically convert your string into prefered format. E.g. (type (edn/read-string "1.2")) => java.lang.Double

also use Double/parseDouble unless you know you specifically need a 32 bit value

i think i would use the phrase "its preferred type" rather than "your".

See: https://stackoverflow.com/a/56536924/172272

I mean “prefered” in platform specific way)

I would only use edn/read-string where I'm expected to consume arbitrary data, if I only accept numbers, I'm going to use something more specific

agreed. if you want a double, do so. (= (/ (Double/parseDouble "7") 2) (/ (edn/read-string "7") 2)) is false

interesting - thanks for tips everyone - this was more involved topic than i expected - since i do not like exceptions - i think i stick to edn-read-string and check the type is what i expect int?

read-string will of course also throw exceptions

Do not use Float

If anything, use Double/parseDouble

You have to work backward from your use case. What are you parsing and what are you using the parsed number for afterwards?

Depending on that you'll be able to know which parsing strategy to use

The choice of parsing strategy is not a matter of style, but of use case. So there is no idiomatic way, all ways are appropriate, but each have their respective use case

really all i'm doing is parsing a port number that someone might send via (System/getenv "PORT") i felt integer should be enough

i expected a parse-int function along the line of this

312 user> (defn parse-int [x]
 313         (try
 314           (Integer/parseInt x)
 315           (catch Exception e
 316             nil)))
 317 #'user/parse-int
 318 user> (parse-int "d")
 319 nil
 320 user> (parse-int "4.5")
 321 nil
 322 user> (parse-int "4")
 323 4
 324 user> (parse-int "-54")
 325 -54
 326 user> 

Again, use Long/parseLong rather than Integer/parseInt (Clojure's "native" types are Double and Long rather than Float and Integer).

Clojure is a hosted language so it expects you to use the host features 🙂

ha good to know - regarding clojure native types - is this not what i think it is?

60 user> (class (int 5))
61 java.lang.Integer
62 user> 

it is. but the defaults are not Integers but longs: (class 5) #_ -> java.lang.Long

got it - but for parsing port number - there is nothing inherently wrong with using Integer is there?

does that create problems elsewhere?

It will be promoted to Long as soon as you use it so there's no real point in trying to "force" it to be Integer.

(Well, depending on what you do with it)

The only time I ever use Integer (or Float) in Clojure is when I'm dealing with Java interop that actually requires that type specifically.

user=> (type (inc (int 5)))
java.lang.Long

makes perfect sense - in this case i just pass it to ring-jetty adapter - but i see - in case i was doing any math at all - no point to bother with Integer

love this channel - thanks all 🙂

(when I first got started with Clojure, I had a tendency to use Integer/parseInt by the way -- because I hadn't gotten used to Long as a default 🙂 )

For that use case, seems edn/read-string would do fine as well

You can probably choose between that and Long/parseLong.

split-with

The most efficient way is going to be not using a vector

yes

a sorted vector is a bit of an oddity

You might be able to get something pretty good if instead of getting rid of entries you are working willing to replace them with something else (nil for example)

It isn't sorted, I initially was tripped up by the wording as well, but the example function is filtering out everything

> (defn cut-vec [vec val] (filterv #(> (get % 0) val) vec)) > => #’fund-performance.core/cut-vec > (cut-vec a 3) > => [[4 6] [7 9]] > (def a[[1 1] [2 4] [4 6] [7 9]]) > => #’fund-performance.core/a > (cut-vec a 3) > => [[4 6] [7 9]]

the example function is filtering out anything, but he does say it's "ordered" by the first element of the tuples. so if that's true split-at would be more "efficient" than filterv. but like you and noisesmith point out there's probably way bigger gains to be had by choosing appropriate data structures

according to my example what do you recommend?

I would recommend (drop-while #(> (get % 0) val) vec)

thanks @U050MP39D, and about the structure, what do you recommend?

I'd need to know a lot more about the problem. if this vector is huge then you could use a binary search to find the split point and then use subvec for instance. but if it's less than gigantic that would be a lot of work for not much reward

It could have around 4000 elements

the tuples are a timestamp and a percentage value

is the timestamp unique?

the timestamp can also be used as a key it that helps

yes, it is unique

also this is for cljs

then I would use a sorted-map

and subseq for your function

thanks, I will do that instead

actually. and this is all academic because 4000 elements is really small. but a sorted vector and then using subseq would actually make sense if you know that the elements will be added in sorted order

actually no ignore that

sorry @U050MP39D, do you recommend me to change from vector to sorted-map or not?

I would definitely use a sorted-map yes

👍

now that I’ve read subseq I understand why vector is very inneficient in this case, but that was the spec they gave me

thanks again