This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-07-26
Channels
- # aws-lambda (15)
- # babashka (7)
- # beginners (124)
- # calva (7)
- # cider (19)
- # clj-kondo (26)
- # clojure (261)
- # clojure-australia (3)
- # clojure-dev (10)
- # clojure-europe (45)
- # clojure-nl (4)
- # clojure-uk (6)
- # clojurescript (10)
- # datomic (7)
- # depstar (7)
- # emacs (11)
- # fulcro (41)
- # graalvm (48)
- # helix (1)
- # honeysql (17)
- # inf-clojure (7)
- # introduce-yourself (3)
- # jackdaw (2)
- # lsp (36)
- # malli (2)
- # meander (2)
- # membrane (1)
- # missionary (11)
- # off-topic (17)
- # pathom (83)
- # polylith (15)
- # re-frame (31)
- # reagent (42)
- # sci (35)
- # shadow-cljs (13)
- # spacemacs (13)
- # sql (19)
- # timbre (3)
- # tools-deps (77)
Random question about :require
and linting.
I’ve got the following code in a namespace:
(ns clj-ns.core
(:require [clojure.string]))
(defn do-stuff [x]
(-> x clojure.string/upper-case))
That works, but it also works when I remove the :require
/`ns` form.
(ns clj-ns.core)
(defn do-stuff [x]
(-> x clojure.string/upper-case))
I get a warning from clj-kondo
about a missing require for the 2nd code snippet, but adding (:require [clojure.string])
removes the warning.
As a general rule, I always use :require
with :as
to alias any namespace, but this behaviour made me wonder whether :require
is really needed if :as
, :refer
or :refer-macros
aren’t used with it :thinking_face:
Are require forms are only needed for aliasing using :as
or loading specific functions/macros using :refer
/`:refer-macros`?@darth10 don't rely on other namespaces loading namespaces for you. in this case you were relying on some other library doing that for you
Thanks! I saw that in this https://github.com/clj-kondo/clj-kondo/issues/339.
But I see the same behaviour with contrib and third party libraries too.
Is clojure.core
or lein
loading all of those for me?
clojure itself is loading it somewhere probably, perhaps in clojure.main when you launch a REPL
Do you know any code / tools / have experience to share about affiliate programs in Clojure / ClojureScript?
I'm trying to read https://clojure.org/reference/vars and it seems to skip the discussion of back quoting, which is another place symbols and vars are manipulated and sometimes interned. Where in the clojure docs is there a good discussion of what backquote does. Is it only discussed in the context of macros or is it discussed independent of macros somewhere?
yes I found that. Is that the only discussion?
What am I looking for? I have never really 100% understand symbols and vars in Clojure. I think most of my understanding is correct, but I know that sometimes I reach the limit and don't really know.
I was just trying to read the specification and digest it. Here are several sentences in that document: 1. Within the template, unqualified forms behave as if recursively syntax-quoted 2. For Symbols, syntax-quote resolves the symbol in the current context, yielding a fully-qualified symbol
a var is an (mutable) object which contains a value, a symbol is a way to get to that value
@U010VP3UY9X Me, 9 years ago: https://stackoverflow.com/questions/9113387/difference-between-symbols-and-vars-in-clojure/9115958
if I type something like this in my code '`(a b c xyzzy/abc d e f)` what is xyzzy/abc
is it a 9 character symbol?
@U010VP3UY9X no, symbols have namespaces and names
note that in my example the list is quoted.
but isn't the reader side-effect-free ?
so then xyzzy
cannot be the namespace, because maybe no such namespace exists. if the reader creates the namespace, it is not side-effect-free
@U010VP3UY9X If you want to understand what syntax quotes expands into, prepend a single quote to the backquote
when the compiler resolves symbols, it checks if the namespace of the symbol corresponds to an ns alias
and what does the reader do with "'(a b c xyzzy/foo)"
if xyzzy exists as a ns and in the case that xyzzy does not exist as a ns ?
user=> '`(+ 1 2 ~@[4 5 6])
(clojure.core/seq (clojure.core/concat (clojure.core/list (quote clojure.core/+)) (clojure.core/list 1) (clojure.core/list 2) [4 5 6]))
@U010VP3UY9X The reader can read from the aliases of the current namespace
is xyzzy/foo
interned? or not necessarily?
!Aha! something I didn't know. only vars and keywords are interned.
true, but in backquotes, the aliases matter.
user=> '`(foo/bar)
(clojure.core/seq (clojure.core/concat (clojure.core/list (quote foo/bar))))
user=> (alias 'foo 'clojure.core)
user=> '`(foo/bar)
(clojure.core/seq (clojure.core/concat (clojure.core/list (quote clojure.core/bar))))
not surprising my questions arise from the experiments with clj-kondo. If I have a macro definition in my def_hook.clj
file, and the macro is defined in the namespace defined by
(ns def-hook
(:require [clj-kondo.hooks-api :as api]
))
but the text of the macro contains something like clojure-rte.util/memoized-multis
what does the reader do? will it fail if no such var has been interned into clojure-rte.util
?or perhaps my question should be, what does the compiler do when it reads the defmacro
does that violate what you said earlier?
you can reason about this purely in terms of the backquote. that it's a macro is just another detail
ahhh, so reading is OK, but when the REPL compiles the defmacro form, it will barf?
So I could call read manually (if I were perverse enough to do so) and manipulate it before passing it on to the compiler.
this is why I said: making the namespace name in the "hook" namespace the same as the original macro ns, will solve this, if you also provide the correct aliases
yes but who converts the text "xyzzy/foo" into a fully qualified name? is it the reader when it encounters the syntax quote, or is it the compiler when it encounters the alias followed by a / ?
@U07S8JGF7, part of my confusion is that I understand this process very will for Common Lisp, and on the surface of surfaces it looks the same in clojure, but many of the corner cases are different.
@U010VP3UY9X That’s right. Clojure does some funky things wrt to aliases (which @U04V15CAJ just explained).
in CL the reader resolves all symbols into namespaces (packages in CL-speak) and the reader fails if this cannot be done. but in clojure the reader does not do this and rather backquote and the compiler share the responsibility in some mysterious way.
you can do it like this in a macro:
(defmacro foo [x]
(let [res `(+ 1 2 3 ~x)]
(prn res)
res))
res
is exactly how the compiler will see itas this understanding is not 100% necessary the beginner to understand. However, more and more I'm limited by my failure to understand.
Reader just resolves symbols. It does not resolve Vars. But alias expansion happens during symbol resolution.
hmmm. I didn't realize resolving symbols was different than resolving vars
A symbol is basically a string. It’s a name which might or might not refer to something.
@U010VP3UY9X You could have a macro expand in whatever symbols like foo.bar.baz/quux
without the namespace / var foo.bar.baz/quux
in existence. Thus you can write macros that generate code for certain dependencies while not actually having those dependencies.
Question. Is all this information really encoded in the text https://clojure.org/reference/reader#syntax-quote if I read it meticulously and carefully enough?
@U07S8JGF7, good point. and a symbol is a data structure which which fields? It probably has a print-name, a home-namespace, perhaps meta-data ?
a symbol is a pretty "dumb" thing, very similar to a string, but it has two fields: namespace + name and metadata
it's not related to any real Clojure namespace (as in the thing that contains vars) "mechanically" speaking
Yes, it has metadata, a name, and a namespace. But it does not necessarily correspond to anything.
can someone find the declaration of the symbol data structure in the clojure/java code and paste it here or paste a link to it?
“namespace” is an overloaded term. There’s a lookup map of vars, then theres the “portion of a name before a /
appears”
https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Symbol.java
also maybe check var: https://github.com/clojure/clojure/blob/clojure-1.10.1/src/jvm/clojure/lang/Var.java
@U07S8JGF7 yes indeed, we use the term namespace for two different things. the text before the slash, and a data structure which is probably an instance of the java this.and.that.namespace class.
and that “actual” namespace is a data structure which holds the mapping of symbols to vars right?
and what does the ns
function do the second time it is called: re-defining a namespace. I.e., in the clj-kondo hooks file can I re-define the namespace form my application?
or perhaps clj-kondo never loads my application, rather it only loads the hooks file?
the ns
macro does not define a new namespace if the namespace already exist, but if new libraries are to be loaded, then it will do that, also it will define new aliases if any
so there's no conflict if my hooks file defines the same namespace as my application ?
since the hook code is not on the classpath (usually, normally, people should not attempt to)
and clj-kondo has to ready my application, and predict how clojure would resolve all the names ?
oh, doesn't it call read
only file and try to figure out which name corresponds to which var by understanding the namespaces and aliases and explicit text before the / ?
yes, it does that with respect to linting, but that is not prediction, just analysis
it has to duplicate the logic which the clojure compiler does right? because both clojure and clj-kondo call read on the UNIX file.
anyway, I'll be away for an of hour or so.
thanks for all the help everyone!
but about the macroexpand feature I'm now iterating on with you: in the hooks namespace, there are no aliases corresponding to what clj-kondo has linted (yet) in the same-named namespace that was analyzed. We just barely got this macroexpand feature working :)
@U010VP3UY9X I also came to Clojure from Common Lisp, and struggled to understand Clojure symbols. Maybe this old blog post of mine will help: http://blogish.nomistech.com/clojure/clojure-symbols-vs-lisp-symbols/ (I’ve only skimmed the above discussion, so maybe it doesn’t add anything to the discussion.)
An added detail, the reader doesn't really track anything, but it runs in-between the compiler, so the environment exists. Clojure will read, compile and evaluate each top level form one after another. That means in:
(ns foo
(:require [foo.bar :as bar]))
`(bar/bazz 10)
The ns form is read, compiled and evaluated first. So now *ns*
refers to the foo namespace instance, and the foo namespace has bar refer to foo.bar
So when unquote is encountered by the reader, it will expand all aliases symbols by looking up in the alias map of the current namespace in *ns*
. Which at that point will exist in the runtime.
If the symbol has its own namespace like xyz/abc, than unquote in the reader won't do anything to it since it's already fully qualified.
Unquote will also apply splicing and all that too.There is also *reader-resolver*
which you can bind to something that resolved aliases in namespaced keywords/maps:
https://twitter.com/borkdude/status/1277633414464712704
So Namespaces (capital N) contains a map of alias symbols to fully qualified symbols, and a map of fully qualified symbols to Vars.
Symbols themselves have a namespace (lower n) part, which is just a string to allow you to have different symbols name in different contexts.
The symbol namespace can refer to nothing, or it can refer to the an alias in the current namespace, or to a real Namespace.
Ok, so what's weird is that this means there is an indirection:
symbols -> Var -> value
But the mapping from symbols -> var
is context dependent, the symbol will first be looked up in the locals bindings, to see if it points to a value there, if not found, it'll be looked up in the current namespace bindings. But if the symbol is namespaced, it will first check if the namespace is an alias in the current namespace and if so it'll expand to the full namespace, and finally once expanded to a fully qualified symbol it will take the namespace of the symbol and look up the corresponding Namespace in the global Namespace map, that will find the Namespace for the symbol's namespace, and then it'll look for the symbol name in that Namespace binding map, to find the Var associated with it. And finally the Var will be dereferenced as well, so that the value is returned.
But unquote doesn't do all that, unquote just expands the symbol to be fully qualified if possible.
Then at evaluation, the symbol will resolve to the Var and the Var to the value, unless you have direct linking enabled, at which point, this will happen at compile time in some circumstances.
Oh ya, true, resolving the symbol to the Var and Var to value is actually done at evaluation time, unless direct linking is on, then it might be done at compile tine
and direct linking only influences direct calls, not when vars are referenced from non-call position
the compiler transforms that symbol reference to code which derefs the var at runtime
findOrCreate will look up the Var form a string, that seems dynamic to me, the Var is not resolved at compile time then
hmm, I can see how this is confusing. The generated bytecode results into a couple of constants that refer to vars
user=> (clj-java-decompiler.core/decompile (fn [] (assoc {:a 1} :a 1)))
// Decompiling class: user$fn__192
import clojure.lang.*;
public final class user$fn__192 extends AFunction
{
public static final Var const__0;
public static final Keyword const__1;
public static final Object const__2;
public static final AFn const__3;
public static Object invokeStatic() {
return ((IFn)user$fn__192.const__0.getRawRoot()).invoke(user$fn__192.const__3, user$fn__192.const__1, user$fn__192.const__2);
}
@Override
public Object invoke() {
return invokeStatic();
}
static {
const__0 = RT.var("clojure.core", "assoc");
const__1 = RT.keyword(null, "a");
const__2 = 1L;
const__3 = (AFn)RT.map(RT.keyword(null, "a"), 1L);
}
}
the assoc
symbol will only be resolved once. you could say that is at compile time, or if you will, at load time, if AOT happened. compile time and load time are sort of intertwined when you don't have AOT
I think load time is more accurate, because it won't fail AOT if missing, so compilation will work, but load will fail
Which parts of clojure emit bytecode directly? I assumed that was the general approach.
The decompiler above automatically converts the bytecode back to Java for convenience
All to say, its "relatively" simple lool, but also there's a lot of little things so keeping track of it all in your head is hard, and so is writing a concise and clear guide on how it works.
Like you can use the eval reader too, and then you can compile and eval things as you read 😛
And you have things like aliases, auto-resolved keywords, data readers, reader conditionals, and all sorts of things like that too
Meh, its out of fashion, but I also love it 😛, even if I don’t really use it for production stuff. Ya lein and project.clj is the most use of it I know
But I like being able to do whatever I want with my languages, even if it means shooting myself in the foot 😛 I have restraint not to go wild when it matters, but when I’m having fun its hella fun.
From http://blogish.nomistech.com/clojure/clojure-symbols-vs-lisp-symbols/, a symbol has a name part and a namespace part. the namespace part might be nil, but if not nil, it names a namespace. However, the namespace need not exist, the namespace of a symbol is just a string which might designate a real namespace or not.
From http://blogish.nomistech.com/clojure/clojure-symbols-vs-lisp-symbols/, this is a real surprise for me (identical? 'foo 'foo)
evaluates to false
whereas (= 'foo 'foo)
evaluates to true
, in CL the equivalent of those two evaluate to true. Two symbols with the same name in the same package are identical objects in CL, but not in Clojure. I'm curious the motivation for this. Was that an oversite in the original implementation, or was it intentional?
This is intentional: symbol identity is contextual, they cannot be considered identical globally
I'm not sure what you mean. (identical? 1 1)
returns true, because there's just one 1
object I guess.
I don't see why there need to be multiple foo
objects? as they are immutable. Well I am supposing they are immutable, maybe they are indeed mutable?
identical?
only returns true if the args are the same object. but you can have many instances of the ostensibly same symbol with different metadata
ahh for purposes of meta data. that would be catastrophic if modifying the metadata one my foo
destroyed your meta data on foo
e.g. (defn foo [^String x])
, there the symbol x
has metadata {:tag String}
. But when I write (def x 1)
it has not.
I think it was intended to be interned at first, and then for meta, couldn’t be, because the API to create them in Java is Symbol/intern
There are some mentions of interning and how macros are different in Clojure vs. Common Lisp in this talk by Rich Hickey: https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/ClojureIntroForLispProgrammers.md
A more brief mention in this talk: https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/EffectivePrograms-mostly-text.md
It seems to me that
implementations are not correct:
(loop []
(let [size (.read input buffer)]
(when (pos? size)
(do (.write output buffer 0 size)
(recur)))))
Read can return 0, and then later return non-zero. It is allowed for stream and readers to return 0 bytes in a read call (e.g. stalled TCP/IP connection). You should only stop reading then -1 is returned, which indicates EOF.@roklenarcic if you post on http://ask.clojure.org, you've got a better shot at either getting a thoughtful answer why this isn't a problem or this has a good chance at becoming a jira ticket
I see this has been discussed before on http://ask.clojure.org
https://ask.clojure.org/index.php/8450/strange-behavior-with-clojure-java-io-copy
Note that some Java libraries DO return 0 on non-0 byte buffer sizes and if you combine slurp with those streams you will get bugs
https://clojure.atlassian.net/browse/CLJ-2533 already a jira issue with discussion on it
What is the idiomatic way of doing graceful stops of threads in clojure? Assuming I have the function
(defn my-thread-fn []
(loop []
(when-not (stop-thread?)
;; do work
(recur))))
and want to communicate the stopping criterion from another thread. Do I use some stop-channel togehter with alts!!
and a timeout chan? Use a promise
? How is this usually done?would you want 100 communications to each thread (each thread having its own, so you could stop individual threads but not necessarily all) or one communication and each thread reads the single stop message? Not sure what your goal is
It does. I am just wondering how people usually do it, as the timeout with the channel approach seems kind of "wasted time".
I'm kind of reading between the lines here, but are you dealing with threads that are processing information from core.async channels? If so, then one useful shutdown pattern is to have each thread shut itself down when the input channel is closed.
it sounds like you are planning to implement polling with alt and a timeout, where maybe you should just being using the poll function form core.async
I like to use a shutdown channel, and often two channels, one channel to signal to process to stop, and another channel for it to signal when it has actually stopped (post any clean up action, etc)
if your process is launched via the go macro or thread macro the second channel is the one that macro returns
Another potentially useful idiom: since go
and clojure.core.async.thread
return a channel that yields the value of its of the go/thread body, you know that once you get a value back from that channel, any loops inside of the body have completed. @hiredman’s approach is good too, it all depends on your situation.
In my current approach I am not using any channels for the data that is communicated to the working thread as I don't want to block and the queue should be able to grow unbounded. So there is no data channel to close.
I usually do something like (loop [] (alt! stop ([_]) channel-where-work-comes-from ([work] (do-some work) (recur))))
It sets a flag on the thread, and interrupts a select group of "interruptable" methods. The javadoc for this is pretty straightforward. Thread.stop is unsafe because it can result in invalid object state or deadlocks, and shouldn't be used for serious applications. https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/concurrent/Future.html#cancel(boolean) https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/Thread.html#stop()
Note that the effects of future-cancel
can be quite surprising - e.g. it doesn't usually interrupt basic IO operations or network sockets waiting for connection:
• https://github.com/jumarko/clojure-experiments/blob/master/src/clojure_experiments/java/threads.clj#L23-L58
• https://github.com/jumarko/clojure-experiments/blob/master/src/clojure_experiments/networking.clj#L31-L63
there is an alt! operation that reads from two channels, one of which is bound to stop
you may want to backup and ask your new question completely, because the previous question included discussion of core.async
Ok. New question: What allows clojure.core/future-cancel
/ (`Future#cancel()`) to work? Is it using Thread#stop()
?
in general cancel on a future is defined in an interface, so anyone that implements the Future interface can do anything when you call cancel
FutureTask is sort of the default implementation and is usually what you will be interacting with, and it looks like it mostly sets the interrupt flag on the thread
> An atom with a flag is one way, stop channel is another
If the thread is known to the caller, would it be OK to use Thread.interrupt()
and Thread.isInterrupted()
?
sure, just depending on the context you may not have a thread, and the thread may not be something "owned" by the code that it is executing at the moment
Interesting. Does the code need to cooperate at all? Or is an exception thrown on the thread that will stop it?
yes that's absolutely correct. here's a specific example improving a clojure codebase in that direction https://github.com/clojure-emacs/refactor-nrepl/pull/319
Ah, I see, so it relies on you calling functions that raise that exception, like Thread.sleep.
I found this neat implementation of the bowling scoring problem in Haskell and wanted to translate it to Clojure.
pins [x,y] = x+y -- last frame
pins [x,y,z] = x+y+z -- last frame
pins (x:y:z:xs)
| x == 10 = 10+y+z + pins (y:z:xs) -- Strike
| x+y == 10 = 10+z + pins (z:xs) -- Spare
| otherwise = x+y + pins (z:xs) -- open frame
As you can see in lines 1 and 2 Haskell can destructure and then dispatch based on how many items are in a list.
What would the idiomatic clojure way be?
I ended up with this so far.
(defn pins [[r r' r'' :as rolls]]
(case (count rolls)
2 (+ r r') ;last frame
3 (+ r r' r'') ;last frame
(cond
(= 10 r) (+ 10 r' r'' (pins (rest rolls))) ;strike
(= 10 (+ r r')) (+ 10 r'' (pins (drop 2 rolls))) ;spare
:else (+ r r' (pins (drop 2 rolls)))))) ;open frame
Some notes:
• Be careful with 'count'. If you pass this function a lazy sequence, it's going to realize the whole thing. Further, it has different complexity depending on what kind of data structure you pass to it. A list or a lazy seq is going to be O(n) for counting.
• You can destructure with &
to get rest
: (defn [[x y z & xs]] ...)
I thus might consider something like:
(defn pins [[x y z & xs :as rolls]]
(cond
xs (cond
(= 10 x) (+ 10 x y (pins (rest rolls))) ;strike
(= 10 (+ x y)) (+ 10 z (pins (drop 2 rolls))) ;spare
:else (+ x y (pins (drop 2 rolls))))
;; last frame
z (+ x y z)
;; last frame
y (+ x y)))
This makes it pretty clear that that original algorithm is undefined for empty and single-element lists, not sure what you want to do there.
might make that more explicit:
(defn pins [[x y z & xs :as rolls]]
(cond
xs (cond
(= 10 x) (+ 10 x y (pins (rest rolls))) ;strike
(= 10 (+ x y)) (+ 10 z (pins (drop 2 rolls))) ;spare
:else (+ x y (pins (drop 2 rolls))))
(and x y) (+ x y (or z 0))
:else (throw (ex-info {} "Undefined behavior!"))))
Thank you Russel, I like the approach to check for xs and z
I'm interested that you didn't use the multiarity dispatch feature that seems to more closely match the original code :
(defn pins
([x y] (+ x y))
([x y z] (+ x y z))
([x y z & xs]
(cond
(= 10 x) (+ 10 y z (apply pins y z xs))
(= 10 (+ x y)) (+ 10 z (apply pins z xs))
:otherwise (+ x y (apply pins z xs)))))
is that cos you prefer to not use apply like this? or is there some other reason that I've missed?The reason was that the haskell version (and the java one) take a list of pins. But yeah, nothing is stopping me from actually doing it that way
@jakob.durstberger core.match
might give you something that resembles Haskell more in this particular case
I didn’t know core.match
existed, it looks pretty cool.
Thanks, I am not unhappy I just wanted to see if I can make it better somehow
You may be interested in https://github.com/noprompt/meander - roughly in the same ballpark as core.match