This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-02-15
Channels
- # announcements (2)
- # aws (3)
- # babashka (2)
- # beginners (100)
- # calva (3)
- # chlorine-clover (22)
- # cider (7)
- # clj-kondo (1)
- # clojure (240)
- # clojure-france (6)
- # clojure-hungary (2)
- # clojure-spec (7)
- # clojure-sweden (3)
- # clojure-uk (19)
- # clojurescript (21)
- # core-typed (8)
- # cryogen (2)
- # cursive (8)
- # data-science (7)
- # datomic (7)
- # duct (7)
- # fulcro (13)
- # keechma (1)
- # luminus (3)
- # malli (3)
- # off-topic (258)
- # pathom (5)
- # reagent (4)
- # shadow-cljs (111)
- # sql (4)
- # tools-deps (12)
- # vim (1)
- # vrac (1)
- # vscode (35)
What are good guides, tips to run migrations on Postgresql? I want to use Migratus (https://github.com/yogthos/migratus)
If you want an instant startup you can try https://github.com/leafclick/pgmig (native-image migratus)
What is difference between using Migratus and native-image?
@U2FRKM4TW how do you run migrations while deploying jar application?
@U2FRKM4TW so migrations are run when you start jar application in production?
I'll be using Dokku (http://dokku.viewdocs.io/dokku/) to run application.
I deploy either self-hosted or Heroku apps, so I don't use jars at all. But it is definitely possible.
And no, you don't have to run migrations on app start, although that probably makes sense - to run them right before running the main app code. But you can create a different main class for migrations. Or you can have a CLI option that runs migrations specifically.
So, migrations run before main application.
@UCMNZLJ93 Migratus can be run as a Java process (or as a part of the Clojure app). Assuming a Linux system used for building then pgmig is a standalone native Linux process (elf binary), no Java or anything else needed. Dockered build is included for cloud deployments.
Migratus is embeddable - you can do anything with it. You can embed it and alter it to your needs. E.g. run migrations offered by unauthenticated web users in real time, if you like having fun. :)
I use Flyway with our postgresql instances. Although there is no clojure specific wrapper for it, writing the interop is pretty trivial.
Hre's a very simple example: https://git.sr.ht/~dharrigan/startrek/tree/master/src/startrek/migration.clj
What's the reason one cannot mutate the value of a dynamic var with set! if it's not already thread-bound?
One reason might be that you want to be assured that the effect of set-ing something is always thread-local?
I'm trying to implement "immutable vars" in sci, so they can be re-used for multiple sessions. E.g. one chatbot session cannot alter-var-root! a var so the next session would be troubled by it. But set!
seems to be safe in that regard.
Which is why I believe its not allowed, forcing you to use the thread safe atomic alter-var-root instead
My impression from Clojure is that it uses immutability only in order to protect from race conditions and data races.
My impression, and several of Rich Hickey's talks if I recall correctly, is that immutability can be immensely helpful in single-threaded programs, too.
e.g. you have nested collections to represent some data. For some sub-collection, you want to insert a new key/value pair, then put that updated one back into its 'parent collection', but not change anything else. Do you need to deep-copy the sub-collection you want to update, or not? In Clojure, never. In languages with default mutable collections, often you do.
If you forget, you may have just modified occurrences of that object used as a sub-collection in N other places, too, without intending to.
No IMO. A data race is only possible in the context of multiple continuations/concurrency.
The issue is more with the programmer. Its hard to realize that the function you passed the collection into will mean that if you use the collection afterwards it might have been modified
When we say x is 3 what we really mean is that x
is an alias to a location which currently holds the value 3
. But there is also a time aspect to it. x
can attain a new value anytime - with or without our knowledge.
I think, by focusing on immutable values and collections, Clojure effectively eliminates this whole class of bugs. None of these is related to concurrency.
But, without concurrency, it is with predictable time, and "without our knowledge" is more, without us realizing, since the code does define a deterministic ordering. So I'm thinking it almost feels like a "Hard to read" bug
What's funny is I have beginners telling me Clojure is hard to read and follow all the time. Yet, I think that's false, compared to mutable imperative programming, its way easier to read and follow.
Once you build things around immutability, the resulting thing is definitely easier to reason about.
But there is also the elegance of LISP syntax. It's an acquired taste I guess, because many (beginners) seem to hate it.
hard vs. easy to read is based on what you already know and are familiar with, and changes over time and learning.
or at least, in some cases it can change.
I don't know a name for such a category of bugs, but it is mutating an object in place, and while in principle it is possible from complete knowledge of a program and its behavior to predict that such mutation is wanted, versus unwanted, is possible, it can be very challenging to keep that straight. It can be a very complex question.
Some people create such 'used/referenced in multiple places' mutable object on purpose, by design, because they want updating the object in one place to make it appear updated everywhere it is referenced. It is keeping the wanted versus unwanted cases separate that can get complex, when the programs and/or data structures are large and have many cases.
This is what I understand to be a benefit of immutability: reading a value, as a programmer/human being, and not expecting it to change. However, Rust allows “shadowing” simply by reusing the let keyword. I asked why it was designed like that and told “if you don’t like it, you don’t have to use it.” Then the conversation went deep into how immutability has nothing to do with not being able to reuse an identifier
Well, depends what we mean by "read". Hard to reason, hard to follow, hard to comprehend.
To me it sounds like those things. Very different from, hard to read because I don't know the syntax/semantics of this language
Seems we're saying there is a class of bugs introduced from mutability making it hard to understand what a variable contains at different point in the code.
I meant everything you said about “read,” and I argued it in that way as well, not just a simple miscommunication
I mean, if someone was programming in assembler for a processor that had 16 general purpose CPU registers named 'r0' through 'r15', and someone said "I as a person have difficulty reading code and remembering what register is being used for what purpose", I would agree that can be a difficulty. But it is something you either accept as part of that language, perhaps using documentation/macros/etc. to make the problem somewhat more tractable for people to understand, or you choose to use a different programming language where the problem does not exist (e.g. C). Mutability of collection in C, Java, Python, C++, Ruby, etc. is at a higher level of programming than register allocation, but it is a thing that a programmer working on a large program can have difficulty with remembering, and/or reasoning about. And immutable data, even in a single-threaded program, is one way to avoid that problem entirely.
I meant read as hard to reason about because you have to track every time a value changes
To me, the hard to reason about as the same identifier shifts values throughout the code, is what is meant by mutable. Apparently not to anyone in the Rust community
Kamuela, I don't know Rust well enough to be sure I know what you are talking about. Is it the same thing as reusing the same symbol like tmp
to name different values, perhaps as an argument to a function, then shadow it in a let
inside of that function, then perhaps a nested let
inside of that shadows it again?
let myName = name; let myName = newName; Allowed ^ even though myName = newThing would cause a compiler error because it’s “immutable”
If that is what you are talking about, yes that can be confusing to a developer trying to understand the code's behavior. It is not a sign of mutability, but of names shadowing other names. You can pretty easily make lint tools that warn you about such shadowing of names, and change your code to avoid using them, if you find it confusing.
I do not know if Rust developers/language-designers consider your Rust code snippet above as an example of names shadowing other names, or something else. In Clojure the corresponding thing would be called shadowing.
I’m not a fan for all the aforementioned difficulties with reasoning about it. Because nothing stops it from happening 150 lines away
And lint tools can be developed (or have been) that can catch it an arbitrary number of lines away
Interesting. Well I’m not gonna die on this hill, just think it solves only a very narrow problem at the cost of opening up a can of worms
I don’t tend to have an issue either inline chaining or renaming intermediate identifiers
Allowing shadowing of names I could imagine being a thing that some language might disallow entirely, but it seems unlikely today. Even Haskell as the "most pure" language allows it, I believe.
Find/use a lint tool that warns you about occurrences, would be my recommendation for removing them from your own code, or others that you are allowed to change, if it bothers you.
It’s hard to believe it does, but I don’t know Haskell so I am not disputing it. Just slightly disappointed haha
This discussion on warnings for name shadowing in a Haskell compiler strongly suggests that Haskell allows it. I am not an expert in Haskell by any means: https://www.reddit.com/r/haskell/comments/5oepn9/better_warnings_for_name_shadowing/
But those two lines could be 150 lines apart ... 🙂
Not saying that is good or recommended style for anyone to use -- just that the language itself won't stop you.
Could they actually? I don’t mean theoretically, but can you shadow later in the form or only in that preamble?
You can have 149 bindings of other names in a single let
before binding the same name again.
You can have let
nested inside of another let
, where the inner let rebinds the same name used in the outer let
, or a function parameter name, or a "global" name earlier in your code.
It is all shadowing of names, according to the Clojure compiler 🙂
There are occasionally bugs that can confuse people in their Clojure code where they use let
to bind a symbol like name
, then try to call Clojure.core/name
inside of that let
body, but don't use clojure.core/name
, but only name
, forgetting that they have shadowed name
The Eastwood linter will warn about such things
Well, at least you believe they are new 🙂
And then I get to use preformattedName later on! Like magic! Not swallowed forever by shadowing
But if the code is already using name everywhere, and there was a bug, and you need to add a trim half way in?
Anyways, I do see your point. I think I can be convinced that shadowing can make things harder to follow and reason about
Cause I feel the same argument can be made to polymorphism. We should have get-vec, get-map, get-list, etc
I wouldn’t make that argument. I love me some magic functions that swallow any argument and do the thing
Depends on the implementation I guess. I tend to assume a magic function isn’t going to trim white space or divide by 100 depending on a subtle argument difference
But pass a collection in and I can imagine whitespace being trimmed on dozens of strings
You have another same debate in Java. They say type inference is harder to reason about. Because it isn't obvious what the type will be
Right, but this is where that line becomes hard to argue where it really needs to be. Does it allow shadowing? Does it allow type inference? Does it allow polymorphism? Overload? Etc.
Hypothetical: Is it possible for your head to overheat, yet your change never or rarely ends up introducing a bug and the actual time to make the change wasn't any longer?
What if your head hurt because of your lack of familiarity, experience and having to learn new things?
Or, the other hypothetical: Is it possible for your head not to overheat, yet your change ends up introducing a bug often, and the change ends up taking longer then expected to make?
I say that, because head hurting to me is the classic hard to read. For example, Clojure makes everyone who don't know FP and Lisp head's hurt when reading it.
But hard to reason I feel is more about whatever you end up thinking this does, most of the time you'd be wrong, even if you knew the syntax and semantics.
If you almost always end up being right about what something does, no matter if your head hurts or not. I feel that's good, then its easy to reason about. Could still be hard to read if your head hurt, but that would either be because of unfamiliarity with syntax and semantic, or because the inherent logic is hard, not accidental complexity
Anyway, that's just me. I think on practice, easy to reason is easy to read as well. And vice versa, once you know the language
It’s accidental complexity because of bad naming. That simple. Shadowing/mutating, to me, is just the same vein as calling something x, mutating it 100 times, and then returning it. When given the tools, people prefer being terse to being clear
Knowing the syntax of a language doesn’t come into it. Knowing the domain of the problem being solved in the algorithm probably comes into it a lot
I don't think you mentioned this, so I'll mention one thing about shadowing when doing a series of calculations that I think is an advantage, for example:
(let [x (some-calc y z 123)
x (if (odd? x) (blah x) (bleh x)
x (+ 23 x)]
...)
The nice thing about using shadowing here, as opposed to using a new name each time (e.g., x
, x'
, x''
) is that it is very clear that the earlier versions of x are no longer needed or used after each step.
I don't use shadowing in other situations because it adds confusion, but in the case I described I think it makes the code easier to think about than if multiple names were used.I can confirm that Haskell allows shadowing, and I think one of the main use cases is the one I mentioned.
Another way of saying it is that you can't accidentally use an earlier version of x because the earlier versions are shadowed.
Ya, I dunno. Like I said I think I wouldn't mind too much either way. Since I've never had bugs due to shadowing that I remember, I feel its pretty safe and easy to reason about. But I don't think just going x1 x2, x3, etc. is a big deal either.
But since I never used a non shadowing lang, I can't say, maybe non shadowing actually could introduce more bugs if I keep accidentally using the wrong "latest" name
If I was reading code that used x1, x2, x3, I would be wasting energy trying to figure out why the earlier versions were preserved and how they were used in the following steps. I'd have to look very carefully to confirm that only the latest version is used.
I feel like you’re also always conjuring up a scenario where your scope never ends. Maybe I’m not in intense enough domains, but I tend to close up scopes in around 300 lines max
I think he is saying that he is so accustomed to shadowing, that avoiding shadowing implies to him that the earlier versions need to be used later for something
And if the code does not use the earlier ones, why did they bother creating new names? It is a confusion introduced not by shadowing, but by different expectations and practices between code author and code reader.
I think making that decision as you’re coding is premature optimization. Truth is I don’t know if I’m going to need the intermediates until I’m done and tested
To make code more readable we can reduce the possible things the reader has to think about. This is just one way to do that. In general I think it's very important to do everything we can to make it readable. My future self is my biggest problem. :-)
I admit, as a shadower myself 😋, I have that expectation. Especially in Clojure. I will just nest expressions or thread them if the intermediate don't need to be used later.
But see, this is where I felt this had overlap with the polymorphism argument, why not get-vec and get-map ?
I can see now in that situation why I don’t like it. It’s because I’m used to using languages that would end up mutating the incoming argument. So I guess to keep my idioms useful for my lowest common denominator languages, I’d be very careful not to touch incoming arguments
I do agree though, with more use of exclusively Clojure and other immutable languages, I would probably relax my feelings about it
I'm willing to change my mind if I start to realize that it is a common source of bug though, or confusion
I think it’s just a smell of other problems/not being conscientious of future maintainers. I think the feature itself, approached with care, is fine. Just used to abuse
I feel shadowing in a let, and shadowing in a nested context might be different too. Actually, I'm not even sure its called shadowing when in a let
Normally, the reason shadowing is not mutation, is because the old value of the variable is restored on the way out
If you mean binding the same name x
multiple times in the same let
, yes that is also called shadowing. This (let [x expr1 x expr2] ...)
is equivalent in meaning and behavior (modulo maybe some minor differences in JVM byte code produced by the compiler) to this: (let [x expr1] (let [x expr2] ...))
OK, yes, the nesting one does let you use the outer binding again after the body of the inner one is done.
But neither involve mutation of data, "only rebinding of the values bound to names", which I agree can seem like a subtle and/or confusing difference.
There is no variable to mutate, is one reasonably correct view on answering that question.
There are two names that are spelled the same 🙂
I would say that a nested let that shadows, and then use of the un-shadowed binding further below, would qualify as abuse of shadowing, because it would make the code more difficult to understand than if two separate names were used. It all comes down to whether it makes it more or less difficult for the reader.
I am almost sure I have written a Clojure fn
inside of a function defined with defn
, where I have without even thinking about it had a parameter to the inner fn
shadow something in the outer scope, which I then used the shadowed-only-inside-the-fn-form name after the fn
body was complete. I don't know for sure that I did this or not, because everything worked with no warnings 🙂
One-line fn
forms have a very tiny scope
I don't know, its normally super obvious. I feel it happens when using generic names. Like the nested form is to be reasoned with independently. So its not always confusing. Like say a nested loop inside a nother, and you use e in both for element, even though they're not going to be bound the same element.
In JavaScript back when CPS was the only way to format async stuff, nested functions’ arguments would shadow upper arguments pretty much constantly. There was no way to do it without potentially doing your err1 err2 thing. Nobody did that
Yeah, I have done it too. But I still think it would be more clear to have a different name, and I've been trying to make myself do that.
I remember having code generating a table of tables. So I definitely had row and col used in some inner nesting, and in the outer one as well 😛
I've been trying to shadow only when I think of it as a pipeline, i.e., where the earlier versions are not used again. In the example with start-time
, I can think of it like that.
Wouldn't be too hard to write a non-shadowing let, I'm curious to try it and see how it feels to use it
By non-shadowing let
do you mean one that gives a compile-time error if the same name is bound multiple times in the same let
? If yes, agreed that should be easy to write as a Clojure macro. If you mean one that does not allow binding of names already in use in the enclosing scope, that would be a bit more challenging.
But, anyway, my point was. If its in a nested scope, it feels much more like polymorphism. Because you still have two distinct variables pointing to two distinct values. Just they share a name. And I can see how that is unlike mutation, but more like polymorphism. That said, inside the same let, if I shadow a name, it feels a lot more like mutation. But, it only "feels" like it, because hold and behold:
(let [a 10
a 20]
a)
compiles too:
final long a = 10L;
final long a2 = 20L;
return Numbers.num(a2);
Maybe still possible to do as a Clojure macro -- I have never dug what exactly goes into the hidden &env
arg of a Clojure macro, but might have the needed data in there.
Yes, just because the "two things" are spelled the same in the Clojure source code, doesn't mean they have to be spelled the same in the code generated by the compiler. They really are two (or more) names. Not variables.
Or another way of saying it: In a default-mutable language like Java/Python/etc., local variables are names of mutable places. In a Clojure let
or function argument names, they are names of values, which have a delimited lexical scope. They are not names of mutable places.
Right, they're bindings.
Here's another way I think of shadowing, just a small point. Sometimes code written with mutation can be simpler to read. One such case is when the prior versions of the variable are not used. Shadowing gives us a way to achieve the same simplicity without mutation. It's a narrow use case, but one that comes up quite a bit for me.
(defmacro letnoshadow [binding-vec & body]
(let [bindings (apply hash-map binding-vec)
global-syms (set (keys (ns-map *ns*)))
local-syms (set (keys &env))
syms (-> (into #{} global-syms) (into local-syms))
shadowed-syms (reduce (fn [acc sym]
(if (contains? syms sym)
(conj acc sym)
acc))
[]
(keys bindings))
bindings (remove nil? (map-indexed (fn[idx e] (if (odd? idx) nil e)) binding-vec))
frequency-of-bindings (frequencies bindings)
shadowed-syms (reduce-kv (fn [acc k v]
(if (> v 1)
(conj acc k)
acc))
shadowed-syms
frequency-of-bindings)]
(when-not (empty? shadowed-syms)
(throw (ex-info (str "Can't shadow existing symbol inside letnoshadow, symbol(s) " shadowed-syms " already bound.") {:shadowed-syms shadowed-syms})))
`(let ~binding-vec ~@body)))
I notice in your macro you take advantage of shadowing by binding shadowed-syms
and bindings
twice. So you'd have to change the names if the real let
didn't support shadowing. ;-)
Exactly! :-)
Hello clojurians! I'm using CLJ-HTTP to make some requests.
I'm receiving a string inside body response, and the header says
"Content-Type" = "application/csv;charset=ISO-8859-1"
And the string is ok but coming with lot's of interrogation marks and numbers (as if it couldn't understand some characters)
and there's no Content-Encoding
header on the response, so CLJ-HTTP isn't doing anything special i think.
furthermore, when i make the same request on the browser the server gives me a base64 encoded string, representing the csv, but with CLJ-HTTP i receive it deserialized.
Ideas?
> Are you going to diverge from Clojure semantics in sci? No, but in CLJS this rule does not apply. I do even enforce it in sci on JS
e.g.:
clojure -e "(def ^:dynamic *foo*) (set! *foo* 2)"
#'user/*foo*
Execution error (IllegalStateException) at user/eval1 (REPL:1).
Can't change/establish root binding of: *foo* with set
$ lumo -e "(def ^:dynamic *foo*) (set! *foo* 2) *foo*"
#'cljs.user/*foo*
2
2
vs:
$ clojure -Sdeps '{:deps {org.clojure/clojurescript {:mvn/version "1.10.597"}}}' -m cljs.main -re node -e "(require '[sci.core :as sci]) (sci/eval-string \"(def ^:dynamic *foo*) (set! *foo* 2) *foo*\")"
Can't change/establish root binding of #'user/*foo* with set
Hum interesting. Without threads, set! is thread safe, so.maybe that's why cljs allows it
well, it's not safe in the sense that the original binding can be changed if you allow multiple expressions to operate on the same dynamic var, e.g. in a chatbot context
Running clj -Spom
, I get Unknown option: "--trace"
even with a blank deps.edn. Has anyone run into this?
Does it happen with other options? Do you have any aliases for clj
set up in your shell? Do you have anything in the users or global deps.edn
?
It doesn't happen with anything else. No aliases. I had a very slightly modified user deps.edn but I removed it and it was replaced with the default
I cannot reproduce it at all.
Do you have strace
? If so, you could use it to figure out what's going on.
The execve calls look fine
execve("/usr/local/bin/clj", ["clj", "-Spom"], 0x7ffbffffade8 /* 64 vars */) = 0
...
execve("/usr/bin/bash", ["bash", "/usr/local/bin/clj", "-Spom"], 0x7ffbffffadd0 /* 64 vars */) = 0
...
execve("/usr/bin/rlwrap", ["rlwrap", "-r", "-q", "\\\"", "-b", "(){}[],^%#@\";:'", "clojure", "-Spom"], 0x565557687ef0 /* 63 vars */) = 0
Neither bash nor rlwrap produce the error in that format. Not sure what else to look at. I've never programmed in CHmm, that java call didn't show up earlier. I was just running strace clj -Spom 2> spom
Works great if I run that java command without the --trace
https://github.com/clojure/tools.deps.alpha/commit/4392e2eb36cb58307f212d1581170189811e2b3b
I'm not sure how to do that, since putting it in deps.edn doesn't help and that's after the last Clojure release.
Hmm.
> after the last Clojure release
So you have the latest Clojure release, installed from that sh
script, right?
I tried that as well
I'm fine with the raw command though, thanks a lot!
what clj version are you using? (you'll see it with clj -Sdescribe
) Did you try clj -Spom -Sforce
? I'm not sure that I can imagine how you're ending up with a bad trace option. I haven't seen this elsewhere.
1.10.1.489, and -Sforce shows the same error
oh, there was a bug there that was fixed in 1.10.1.492
https://github.com/clojure/brew-install/commit/b9d3feb636b52d37a826e1306e70ddf59322ebdb
I forgot about that one, but you should update
latest (yesterday) is 1.10.1.510
I'm on Ubuntu actually, is that just Mac?
Oh I see the script
Okay, I didn't know about that. clj -Spom
works with 1.10.1.510. Thanks!
brew
is available on Linux and that's what I use to help stay up to date on clojure
, since I can just run brew upgrade clojure
periodically (on both macOS and Linux).
(and on Windows I use scoop update clojure
to stay up to date)
Thanks, Sean. That is definitely easier.
True, I don't know how to call that. I guess its just the programming style makes order hard to reason about in that case. But the behavior should still be deterministic, so its not a "race" behaviour.
has anyone written up anything about using deps.edn with https://github.com/features/packages ?
deps.edn
can both use an explicitly specified Maven repository and a Git dependency that itself has deps.edn
file. Not sure there's anything more to it.
thanks, not what I was asking - oddly github packages can be really finicky with authentication. 🙂 so was going to write something up about it but didn't want to be redundant. worth checking out the link, github packages is using maven (but in their own weird form), not a direct git dependency.