This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-08-01
Channels
- # announcements (1)
- # beginners (71)
- # calva (12)
- # cider (3)
- # clj-kondo (12)
- # cljsrn (9)
- # clojure (167)
- # clojure-dev (21)
- # clojure-europe (5)
- # clojure-italy (60)
- # clojure-nl (6)
- # clojure-spec (6)
- # clojure-uk (60)
- # clojurescript (86)
- # core-async (25)
- # cursive (21)
- # datascript (9)
- # datomic (4)
- # figwheel (8)
- # fulcro (7)
- # graalvm (4)
- # jackdaw (6)
- # jobs (8)
- # juxt (2)
- # leiningen (1)
- # malli (1)
- # off-topic (20)
- # re-frame (31)
- # reagent (5)
- # remote-jobs (3)
- # shadow-cljs (57)
- # sql (6)
- # vim (9)
anyone here have knowledge in image processing ?
Hello, i’m new to clojure and I’m having some trouble. I need to test s3 responses and what data is in the bucket. What’s the recommended way to do this? I’ve come across amazonica but i’m having trouble. Do i need to create fixtures? I’m also not sure if i need to pass in credentials.edn which has AWS secret key and access key or if i make up fake keys. Any help would be greately appreciated
There's also aws-api from Cognitect (https://github.com/cognitect-labs/aws-api) which can be a newer interesting alternative to amazonica
@mirkopalancaji Hi and welcome to Clojurians! If you're brand new to Clojure and have questions about the basics, the #beginners channel has a lot of people who've opted in to helping out with new folks. If you've got any questions about navigating Clojurians, feel free to ask in #slack-help There's an #aws channel that may help get you answers to AWS-specific stuff. I looked for an Amazonica channel but I don't see anything specific for that.
If Amazonica has a "setup" function, test fixtures are probably the way to go to set up the environment or context for your tests (I'm not familiar with that library).
You might also want to look at the Cognitect AWS library since that provides a nice, thin wrapper over pretty much the entire AWS API.
Ah, I see @jumar suggested that. Cool. Thank you.
Thanks for the information @seancorfield @jumar. I’ll definitely take a look and research further 🙂
There's also a #testing channel if you need to deep dive on testing approaches in general.
Hi all, I'm building a dev tool using Clojure and would like to write configuration files for the tool in Clojure (similar to leiningen or riemann I suppose). I want to read those configuration files, which may include multiple forms, in a safe way. The files will have been written by the user of the tool so should be safe in terms of their content, however I'm wondering if I should be doing any validation of the contents before evaluating them. What's the best approach to do this?
Additionally, how would I go about making certain bindings available for the evaluation of those configuration files? I had a look at how leiningen does it but need to dig deeper to understand it fully.
Try something like:
(binding [*ns* *ns*]
(in-ns sym)
(require ...)
(intern *ns* 'foo (fn [] ...))
You could also try clojure.tools.reader
. If you want an AST, there's clojure.tools.analyzer
.
@ULML9PYJH Thanks. I'm a bit confused by that. What's the purpose of the binding
call?
Hmm, is it so that clojure.core
functions continue to resolve after the call to in-ns
?
https://clojuredocs.org/clojure.core/*ns* > ns is a root var and in-ns calls set!, which only works after someone somewhere calls the binding macro
I did have to require-clojure
inside the new ns
to be able to use any clojure.core
stuff, not entirely sure why
Hmm, from the REPL I can call in-ns
outside of a binding. It finds or creates that ns and switches to it.
However, in the created namespace, there are no aliases of clojure.core
vars
does it make sense to treat a collection as a virtual reduction and invoke xform on it directly and get a virtualized collection without recourse to transduce?
(xform (virtualize collection)) returns
virtualized-collection
xform in the context of transducers refers to a function that transforms a reducing function, it takes a function of the form (fn [accum item] accum)
and returns a new function of the same form, so applying that xform to a value and not a function is GIGO
(virtualize collection) is supposed to return a function that knows about the values of the collection
forget about the previous question.
how do I turn ((map str) (eduction (range 10)))
into a collection?
(eduction (map str) (range 10))
is valid for example, because internally it has a reducing function, something like (fn [accum item] ...)
which internally it applies (map str)
to
(transduce (map inc) + 0 (range 10))
is more or less (reduce ((map inc) +) 0 (range 10))
given an xform and a collection, the easiest way to produce a new collection is using into, you just supplied an empty new collection and the input collection is poured in via (xform conj)
and you don't have to think too much about it
I cannot, but if anyone can I think it would be @cgrand (someone, back when transducers were introduced wrote an email trying to figure out why the 0 arity existed and what it was used for, and I don't recall the outcome of that, and it might have been cgrand)
if recall in most cases the 0 arity of the transducer is basically ignored and the 0 arity of the original reducing function is used
It’s rather; you can’t rely on the 0 arity to be called, and you can’t do more in it that call the downstream (rf)
Is it ever called? That’s been a long-time puzzle from my p.o.v.
😆 Thank you for looking! It's definitely one of those things where you know it's the right thing to do (define a zero arity version) but you're never sure why you're doing it 🙂
so it isn't something you can rely on, which is why the only sensible thing for it to do is forwad the call on unchanged
I haven't yet run into a single case where I've seen the zero arity version called but that sort of makes sense.
transduce never calls it, most other "reduces" don't give you the option to leave out the initial value
Is there a concise way to get the file from a namespace symbol? I have an idea of how to do it via clojure.tools.namespace but it seems kind of involved.
like, there is no guarantee that a file exists, or that a clojure source file exists
(a single file can contain multiple namespaces too)
Ah, yeah I’m not so worried about those cases, i’m just looking at the tools available in tools.namespace
Clojure's own require
uses a private function root-resource
to turn a namespace into a Java resource path. You can call private functions from outside of the namespace using syntax like this: (#'clojure.core/root-resource 'my-ns.core)
(which returns the string "/my_ns/core"
Does anyone know if there is a difference, under the hood, between using ->ARecord
vs. ARecord.
for instance creation? We have some old ARecord.
usages lying around and I’ve been seeing some rare errors running on JDK 11 that I suspect are related to records…
Naively I assumed that the difference was largely syntactical
Mmm that is good to know
also if you use ARecord. you are extremely likely to forget to require the clojure namespace instead of just importing the java type
So the ARecord.
syntax isn’t only deprecated, it’s potentially more dangerous?
using ARecord. is not deprecated. This is normal Java constructor interop and is perfectly fine for arbitrary Java objects (and even "ok" for records). But it is strongly preferred to use the Clojure native means of construction (which also means you don't have to import the internal impl class).
> This is normal Java constructor interop and is perfectly fine for arbitrary Java objects
Makes sense! I was referring to “deprecated” in the context of records, since there are no ->Class
constructors for plain Java classes.
In the context of deftype
s and defrecord
s, what implications might using the ARecord.
syntax have wrt reflection / classloading?
or at least not anything obvious to me. if you're using records, then you're in "Clojure world" and reflection is not something you'll encounter - you'll always be driving through Clojure abstractions like IPersistentMap, ILookup, etc or through protocols or interfaces for inline impls. I would recommend never making Java interop method calls on records, but if you did I guess you might have the type for a constructor but not if using the constructor method. Since you should never do that, I wouldn't expect it to be an issue.
for class loading, you're loading the class either way
> I would recommend never making Java interop method calls on records
in this case referring to e.g. (.afield record)
vs. (:afield record)
?
consider the case where your type/record implements a java interface, for the purposes of calling that method from Java
does that change things in any way?
all record fields are private, so (.afield record)
won't work anyways
actually, maybe they are public, but you should strongly prefer using normal map keyword lookup (this is special-cased and optimized for records), and it's portable to cljs
I ask because I’m seeing a (very infrequent) compiler error during our application warmup wherein we’re loading a Clojure namespace from Java, calling a function that will return a type, and then invoking methods on that type. In the course of requiring these namespaces (through the Clojure api), we get an error during record compilation that it can’t find a particular symbol
for java interfaces, you'll want to typehint to the interface, but that's likely true regardless of how you constructed it
The two occurences of this happen while evaluating an (ARecord. field1 field2)
call
And only happens on JDK11
what's the error?
well if you're calling all this stuff from Java, that's a whole different set of concerns
Caused by: java.lang.RuntimeException: No such var: a-namespace/a-function
We’ve used this pattern for a while and I’ve never seen this happen on JDK 8
But does happen rarely on JDK 11
I feel like we've kind of backed way into this problem description, which probably has nothing to do with records
it may not. It’s a theory because I can’t find a way to force this to reproduce
based on that description, I'd say it's most likely a timing issue and 11 happens to load faster than 8, exposing the issue (which is probably there even in 8)
ns loads are not atomic and concurrent loads from different threads can cause issues like this
where two threads are racing in loading the same ns
any chance you've got multiple threads loading code at the same time?
I was suspecting concurrent loads as well initially too.
We shouldn’t; the namespaces are required serially
And in addition, this happens while we’re issuing warmup requests against our application and we fire those off 1 at a time
So I can’t rule it out 100%, but based on how things are set up there should only be 1 thread loading at a time.
These namespaces are also loaded during the instantiation of a singleton that follows the LazyHolder
pattern
Ah look at that
this is something we've been stepping towards fixing recently
there is a (private) var in core now called serialized-require that uses a loading lock to avoid stuff like this
not public yet as we're hoping to get rid of it and just make require have those semantics. but if you were in a position to repro it, you could use that instead of normal require and see if it fixed things.
Hmm interesting… I wonder why LazyHolder doesn’t fix this in this context whereas synchronized
in the issue you linked does
The JVM is supposed to guarantee thread safety there
it should, but maybe worth validating your code
Hmm yeah
Our code looks like this -
private static IFn A_FIELD_1;
private static IFn A_FIELD_2;
private static class LazyHolder {
private static final AClass INSTANCE = new AClass();
}
public static AClass instance() {
return LazyHolder.INSTANCE;
}
private AClass() {
try {
String namespaceOne = "";
String namespaceTwo = "";
// Throws in here
ClojureUtil.requireNamespaces(namespaceOne, namespaceTwo);
A_FIELD_1 = var(namespaceOne, "a-function");
A_FIELD_2 = var(namespaceTwo, "another-function");
} catch (Exception ex) {
log.error("Exception while loading Clojure namespaces", ex);
}
}
In case you’re curious :man-shrugging:
Maybe just making that util requireNamespaces
function synchronized would work
All of our namespace loading from Java goes through that right now
is this the only class you have doing this?
btw, looks like A_FIELD_1 and 2 should be final too
That’s very helpful actually
Time to go update those constructors haha
thank you 🙏
I suspect these old constructors are the root of some of our JDK 11 woes
i have a namespace split across a bunch of files using in-ns
. i'm finding that test-refresh
doesn't reload the changes properly. has anyone gotten this to work together?
while that is a thing you can do, and it is something clojure.core does, it is not common practice so most tooling doesn't support it
I would have structured this differently if I had been the one to initially design it
follow-up, then! I have a file that's in part of (in-ns 'game.core)
namespace. I have a bunch of functions that are pure, and some !
functions that call the pure versions and apply the results to the state atom. I can't pull the whole file out into a new namespace because of circular dependencies: functions elsewhere in game.core
call the !
functions, and the !
functions call other game.core
functions (it's a giant mess, lol)
in situations like this, what's the best method for making this less messy? should I pull the pure functions into a new namespace, and then require them? Should I pull out the stateful functions into the namespace, and have them take the needed functions (`update!`, etc) as arguments?
if the answer is "who the heck knows", that's valid too! this is a plate of spaghetti, and I'm mostly fine with it, just annoying to test sometimes
introducing proper namespaces with pure and sideffecting functions sounds like the way to go
whatever turns spaghetti into ravioli is always a good call, then can be refactored later
while the namespace system precludes static circular dependencies you can have circular dynamic dependencies
oh really?
do you have an example?
or do you just mean using declare
?
bye mitch 👋
(ns a)
(defn f [g]
(fn [x] (g x)))
(ns b
(:require [a]))
(def f (a/f inc))
(ns c
(:require [a b]))
(def f (a/f b/f))
is this an okay way to write this?
(ns game.stuff)
(defn break-sub!
[update! state side ice sub]
(update! state side (break-sub ice sub)))
(ns game.core
(require '[game.stuff :refer [break-sub!]])
(defn update! [blah] (blah blah))
(defn break-sub!
[state side ice sub]
(break-sub! update! state side ice sub))
@hiredman that's really good to know, thank you!
That last function is a recursive call to itself -- not what you intend, I suspect?
Instead of :refer
use :as stuff
and then call (stuff/break-sub! ,,,)
oh yes, that's what I meant, thanks
But I'm not sure why you'd bother with a wrapper like that? Why not just call stuff/break-sub!
directly?
it is often the case that people are bad at separating the runtime image/structure of their programs from the static structure of their source code
which is one of the reasons I like to use component, the static structure of the source code is what it is, and the runtime image is defined by component
as an example (this happens all over the place, lol), multiple places in game.core
call break-sub!
, and break-sub!
relies on a function from game.core
. I don't want to have to pass the game.core
functions into every break-sub!
call. I didn't know I could reference a function from another namespace without requiring it, tho
thanks for the input, i'll have to play around with this some more!
referencing functions without requiring them as an end run around circular namespaces dependencies is a bad idea
you have to require somewhere to load the code into the Clojure RT (via classloading)
expecting that to be done implicitly is a bet you will eventually lose
that makes sense
for example, sometimes people call clojure.string/whatever without explicitly requiring clojure.string. that works, but only because Clojure bootstrapping happens to load clojure.string. If we optimize bootstrapping to not do that, then this code would break.
it doesn't make much sense to me to use evil with paredit, either you are in one mode, or the other, no? i just switch manually
I use evil-smartparens with strict mode + evil-cleverparens Works well!