This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-12-18
Channels
- # adventofcode (9)
- # alda (2)
- # babashka (89)
- # beginners (86)
- # bristol-clojurians (1)
- # calva (54)
- # chestnut (2)
- # cider (18)
- # clj-kondo (19)
- # clojure (160)
- # clojure-brasil (2)
- # clojure-europe (6)
- # clojure-italy (8)
- # clojure-nl (46)
- # clojure-spec (52)
- # clojure-uk (45)
- # clojured (4)
- # clojuredesign-podcast (11)
- # clojurescript (15)
- # core-async (50)
- # datomic (41)
- # devops (3)
- # emacs (10)
- # fulcro (95)
- # graalvm (11)
- # graphql (3)
- # kaocha (1)
- # luminus (1)
- # malli (4)
- # off-topic (13)
- # overtone (3)
- # pathom (8)
- # re-frame (13)
- # reitit (10)
- # shadow-cljs (83)
- # spacemacs (2)
- # specter (9)
- # tools-deps (3)
- # vim (2)
I get uncomfortable about using :type
or :ns-qualified/type
as a map key, because it means that destructuring it using :keys [type]
would locally shadow the clojure.core/type function. Is this a justified concern or something people regularly do? Having to do something like {typ :type}
each time is even clunkier.
It is a justified concern, IMO. I’ve been bitten far too many times by this avoidable mistake.
(I also usually work in an editor that highlights un-resolved symbols, and I put a lot of trust into it.)
@qythium I think there are certain code function names that people regularly shadow. I don't worry about shadowing type
since I almost never use it in production code (just in the REPL while exploring).
I think name
is another thing that people regularly shadow. And if you're destructuring a map, you can always use the form that maps keys to different symbols (although I don't like that much).
If your functions are short, shadowing core names isn't really a problem -- the context is obvious (fits easily on the screen) and you're unlikely to have both invocations of the core function and usage of your local binding that shadows it in the same function, since you can delegate to another small function with an argument name that doesn't shadow the core function you plan to use on it.
While I wouln't oppose shadowing for the sake of it, it's also true that shadowing code can be a bit fragile; as code changes over time, you might remove or break the destructuring, and type
would suddenly refer to clojure.core's
So I tend to {user-type :type}
, which isn't as ugly as {typ :type}
(`user` being some business-specific concept)
Aside from type
and name
, suspects include identity
, key
... the list might go on, so for me it's easier to avoid shadowing altogether than keeping a mental list of acceptable cases
https://github.com/jonase/eastwood provides automated shadowing analysis btw!
I feel like i'm missing something with clojure.java-time.... does anyone understand why this returns 2020 as the year? I'm very confused...
(defn change-ds-format [in-ds]
{:pre [string? in-ds ]}
(let [input-format "yyyy-MM-dd'T'HH:mm:ss"
output-format "MM/dd/YYYY hh:mm:ss a"]
(->> in-ds
(jt/local-date-time input-format)
(jt/format output-format))))
(change-ds-format "2019-12-29T00:00:00" )
;=> "12/29/2020 12:00:00 AM"
;; Why is this one 2020 ???
This is a wild guess as I don't remember the date format strings: is it correct to have the year the input yyyy
and YYYY
in the output?
Hi, Can someone recommend good resources on building scaleable and resilient Clojure apps, with modern CI/CI, Kubernetes deployments etc.? I can't seem to find much on the subject.
I can’t really recommend it, since I haven’t looked at it yet. But it’s atleast on topic: https://github.com/resilience4clj
Recently i subscribed to https://learning.oreilly.com/home/ which offers a free trial. You can find all (?) Clojure books over there, including Microservices with Clojure
(which piques my interest, haven't read it though)
most of this will be the same instructions you follow for java - you are deploying a library that runs on the jvm
Thank you all! @U051SS2EU This is about the same conclusion I am drawing: create a modular app and deploy it to Kubernetes as you would with a Java app
https://polylith.gitbook.io/ (from -minded folks) has some thoughts on modular apps I think it influenced me (I do modularity) but don't remember tbh
@dennisa What exactly are you looking? ci/cd and k8s are independent things from Clojure and you wouldn’t find anything as Clojure k8s or something like that.
I guess I am looking for best practices on architecture and deployment, and how do they play together . Most of the articles I was able to find involve ring or luminous on the servers side, and they end with basic examples. Re deployment, most of the stuff is about deploying java apps with docker. I am not sure if these all are up to date and reflect the modern Clojure vision.
AFAIK if you make an uberjar of your Clojure app, then the deploy and ops aspect of that app is exactly the same as any Java app, and so all the best practices would apply.
There are good arguments as to why not to make uberjars and instead run your app with the clojure
CLI program. Even still, most of the things are the same.
It looks to me like DevOps is moving towards some of the same ideas that underly Clojure. Immutable architecture and GitOps in particular is functional programming at scale IMHO.
As for concrete tutorials, there definitely aren't as many that talk about Clojure as there are other languages.
Sorry for being so wordy 😅
You have to get experience from ci/cd, k8s (DevOps) and Clojure separately and connect it by wisdom. Seriously, you wouldn’t find anything deep in google about that,.
Here I have an example https://github.com/kwladyka/fy-stock/blob/master/Dockerfile
but it is not like a special choices for Clojure, it doesn’t matter what language you will use
Building a Docker image can be super simple with JUXT's pack.alpha. Thought I would mention it.
hi all, i am having a problem with re-find
- on some files it just seems to hang forever and nothing happens.. here is a way to reproduce the issue:
;; here is my problem
;; using Clojure 1.10.1
;; download this file and untar/zip it
;;
;; you need to update the form below to point to the babel.min.js file
;; in the directory where you unzipped the archive
;; according to `file`, this file is:
;; babel.min.js: UTF-8 Unicode text, with very long lines, with no line terminators, with escape sequences
(def line
(first
(with-open [r ( "/tmp/mark/package/babel.min.js")]
(doall
(line-seq r)))))
(def re #".*[Cc]opyright.*(19|20)[0-9][0-9].*")
;; when you run this form, it will hang the repl/program and never return
(re-find re line)
;; for contrast - if you run this on the file `babel.js` instead, it will
;; return quickly (with no output) which is correct
i did take a thread dump, and the interesting part is this:
"Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007f669c08c800 nid=0x8be in Object.wait() [0x00007f6669704000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0000000719608ed0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
- locked <0x0000000719608ed0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216)
"Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007f669c08a000 nid=0x8bd in Object.wait() [0x00007f6669805000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0000000719606bf8> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
- locked <0x0000000719606bf8> (a java.lang.ref.Reference$Lock)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)
"main" #1 prio=5 os_prio=0 tid=0x00007f669c00a800 nid=0x8b3 runnable [0x00007f66a27f2000]
java.lang.Thread.State: RUNNABLE
at java.util.regex.Pattern$BmpCharProperty.match(Pattern.java:3799)
at java.util.regex.Pattern$Curly.match0(Pattern.java:4274)
at java.util.regex.Pattern$Curly.match(Pattern.java:4236)
at java.util.regex.Pattern$Start.match(Pattern.java:3463)
at java.util.regex.Matcher.search(Matcher.java:1248)
at java.util.regex.Matcher.find(Matcher.java:637)
at clojure.core$re_find.invokeStatic(core.clj:4905)
at clojure.core$re_find.invokeStatic(core.clj:4898)
so it looks like it is stuck in the match
methodI could check this myself soon, but out of curiosity, how long is the string you are doing re-find on?
And have you tried leaving out the ".*" at the beginning and end of the regex?
If you have individual text lines that are tens or hundreds of megabytes long, then the ".*" in the middle of the regex might become an performance issue, but that would surprise me.
Ah, I see you already answered the question before I asked about having very long lines.
its 1654599 chars
good point - let me try removing the .*
and according to text search using the 'less' command, does not contain any occurrences of the string "opyright".
that was the problem - the .*
thanks!!!! 🙂
Basically you were asking it to match the entire string of 1.5 million chars, which I'm a little surprised hung it, but certainly it seemed to be unlikely what you needed.
It does seem like a possible performance bug that JVM devs would want to know about, or perhaps already do
thank you very much for your help 🙂
i ❤️ the clojure community
Thinking on it a bit more, likely that regex starting with the greedy ".*" causes backtracking, i.e. every time it finds a c or C in the string, it checks to see if it can complete a match from the beginning of the entire string. If not, try one shorter. If not, try one shorter, etc. until finally a 0 length string before the c or C also does not match, and only then does it give up and find the next c or C character, and start that whole process over again.
Probably a non-greedy ".?" at the beginning would also avoid that, as would anchoring the beginning of match to the beginning of a line using a regex starting with "^."*
removing the ".*" at both ends was fine, and i still got the behavior i needed
Weird situation with a protocol: I have the protocol definition
(defprotocol API
(get-db [this request table])
(get-model [this request table])
(get-role [this request doc]))
and later in the same file this record implementing the protocol
(defrecord Site [db-f role-f models]
API
(get-db [this request table]
(db-f this request table))
(get-model [this request table]
(first (filter #(= (:type %) table) models)))
(get-role [this request doc]
(role-f this request doc)))
I have a test suite for this library, and everything works great when I run the tests. But when I include the library in another project and use it, I get
java.lang.IllegalArgumentException: No implementation of method: :get-model of protocol: #'cchesty.api/API found for class:
I would think that maybe old code is being loaded somehow, but there is no old implementation of Site. I get the same error with CIDER and via lein run
, so I'm out of ideas.Only for uberjar... but after I ran lein uberjar, the error went away, so maybe the implicit lein clean did the trick? Should have thought of that, but I still don't understand the cause
my guess is you are importing the type created by defrecord without requiring the namespace that creates the defrecord somewhere
I strongly recommend never importing the type and always use the constructor functions generated by defrecord since you are less likely to make that kind of error
similarly I strongly recommend against aot compilation because it can hide those kind of errors and manifest them has hard to understand and debug errors
lein clean && lein run works fine. The import isn't the problem here (almost everything relevant in one ns for dev) but that's really good advice. I forget about the ->Record constructor
Saves a line off the ns form too, nice bonus
Working great now, thanks 🙂
there are other ways it can fail too, that have to do with compilation order, the way lein aot compiles doesn't match with how the compiler works entirely
There is only one import outside of the tests, and it requires the ns also
(ns cchesty.models
(:require [cchesty.acl :as acl]
[cchesty.api :as api]
[cchesty.db :as db]
...etc...)
(:import ))
The only :aot I have is under :uberjar, although I don't know all the details of how lein works be certain nothing is being aot'd during devthe classfiles end up in target/classes or something and that is always on the classpath
Is it not more efficient for deployment?
Oh, startup time isn't a big concern. I'll see if I can run it without AOT
clojure always compiles to bytecode before running anything, aot compilation just tees that off to disk to be re-used
Got it packaging without AOT. Thanks, I learned a lot.
I'm using S3 as a repository, via leiningen
Is this letfn
-ish macro conceptually correct?
(defmacro letfn2 [fnspecs & body]
(let [syms (map first fnspecs)
state-sym (gensym "state")
fns (map (fn [sym]
`(fn [& args#]
(apply (get (deref ~state-sym) '~sym) args#))) syms)]
`(let [~state-sym (volatile! {})
~@(interleave syms fns)
~@(mapcat (fn [sym fnspec]
[sym `(fn ~@(rest fnspec))]) syms fnspecs)]
(do ~@(map (fn [sym]
`(vswap! ~state-sym assoc '~sym ~sym)) syms)
~@body))))
It seems to work:
(letfn2 [(f [x] (g x)) (g [x] (inc x))] (f 10))
Just wondering if I haven't missed anythingthe fns are "named" anonymous fns, so I'd expect (fn ~some-name [...] ...)
- this effects self-calls that are not recurs
(but using mutation will be more similar to the built in letfn for things like function identity)
(defn vswap!*
[vol f & args]
(.reset ^clojure.lang.Volatile vol (apply f @vol args)))
right, volatiles make no attempt to be thread safe, that's the point of volatiles (moved this into a thread from a top level comment because of conversation flow)
This isn't related to that. This is about doubling side effects and improperly written macros.
but that was defn, not defmacro
or do you mean clojure.core vswap has this problem?
For comparison this is the macro:
(defmacro vswap!
[vol f & args]
(let [v (with-meta vol {:tag 'clojure.lang.Volatile})]
`(.reset ~v (~f (.deref ~v) ~@args))))
would:
(defn vswap!*
[vol f & args]
(vreset! vol (apply f @vol args)))
have the same effect?I would not be surprised if maximum efficiency was a goal, given the kinds of places volatiles are used in transducer implementation.
@dominicm I bet there isn't a single place where vswap!
is used like that, since you'll lose the reference to the volatile!
(but still you're right)
consider (if (some-test-with-side-effects?) v0 v1)
- I could see that being used inline
(ins)user=> (def v0 (volatile! 0))
#'user/v0
(ins)user=> (def v1 (volatile! 1))
#'user/v1
(ins)user=> (vswap! (if (println :foo) v0 v1) inc)
:foo
:foo
2
the bug happens here even without the creating expression
:thumbsup:
@dominicm if you don't want to make a jira for the above I can probably do so today - that's the closest to a non-contrived version of the behavior I can think of at the moment
or "fetch volatile via non-idempotent API call"
(vswap! (we-can-do-this-only-once-or-Apollo-13-wont-be-able-to-make-it-back-to-earth!) inc)
I think volatile!
and vswap!
are just bad. introduced to squeeze the last bit of performance out of transducers, but I wouldn't use them
vswap! is just silly, it acts like it provides clojure's time model over volatile! cells, which it absolutely does not
i think people are too eager to reach for volatile!, but I don't think it's "bad", and it does exactly what the docstring says:
"Non-atomically swaps the value of the volatile as if:
(apply f current-value-of-vol args). Returns the value that
was swapped in."
Use atoms for concurrent mutation, use clojure.lang.Box otherwise (Edit: for clarity since people keep misreading this, I mean if you use multiple threads at all, and your mutable reference is visible to different threads, use an atom. full stop, multiple threads, use an atom. If you have a single thread, there is really no need for mutation you can write a nice functional loop and do whatever you want. However, if you insist, do not use volatile! cells for that.)
Box is not public api
there are approx 0 good reasons to ever do unsafe concurrent mutation
I am not sure how to read that as a commentary on my suggestion to use atoms for concurrent stuff
I read it as "otherwise" == unsafe concurrent mutation
but maybe you just meant mutation with identity
(and in that case, I'd suggest an object array instead)
I just meant whatever silly non-concurrent mutation people end up doing where they could use loop/recur
like somewhere above in the channel there is a reimplementation of letfn using volatile! cells to tie the knot
a valid use case for unsynchronized concurrent mutation : lazy memoization of a referentially transparent expression, cf java.lang.String/hashCode
Dunno. Even if a single thread ever mutates a the val
of a Box (https://github.com/clojure/clojure/blob/653b8465845a78ef7543e0a250078eea2d56b659/src/jvm/clojure/lang/Box.java#L17), since that val is not volatile
(Java lang keyword), other threads are not guaranteed to see those changes
(JCIP stuff)
So Clojure's volatile
seems like life insurance
what does Box
offer compared to volatile!
except that it isn't a public API and doesn't work in CLJS?
box doesn't pretend to be something it isn't, and the fact that it is so weird should make you step back and think if you need mutation in that situation at all
you should rank not doing this at all well above using Box
I read somewhere that this is not entirely true. Volatiles do not bypass cache for performance reasons. Instead it uses a tagging system to maintain cache coherence.
I can imagine some CPUs having mechanisms now to synchronize the caches between cores within themselves that might be faster then going to ram
In such case it might still use a CPU cache. The point being the volatile guarantees read-after-write consistency
I have an app that is reliably segfaulting on kubernetes, but not locally using the same docker image. Is there a way to make the SIGSEGV error report print to stderr so kube logs pick it up?
@arohner you might be able to do it like this? https://github.com/clojure/clojure/blob/653b8465845a78ef7543e0a250078eea2d56b659/src/clj/clojure/repl.clj#L280
is there a notion in spec similar to cut in prolog to prevent backtracking? The problem being that if you fail some spec it could be marked as the correct failed spec? It seems if there are several possible specs for something, to mark it as a failed one of the several failed specs rather than failing all of the specs?
Can't answer your question, but there's #clojure-spec. Maybe someone in there knows the answer.
I wonder if either a) you could control its location or b) you could have a sidecar that ran cat *.err
on a loop
at least on adoptopenjdk 13, without that option, it writes to /dev/stdout when I kill -SEGV
the process
Yeah, the contents of stdout on sigsegv are much less interesting than that hs_err file