is there a way to list the signatures of a protocol?

Does anyone know of a short reference for all the java concurrency tools

types of queues, countdown latches, etc?

Hello, I’m probably missing something here, any idea ?

(refer 'clojure.core :exclude '(val))
=> nil
(def val 1)
WARNING: val already refers to: #'clojure.core/val in namespace: user, being replaced by: #'user/val
=> #'user/val

Hello ! I'm runnig windows 10 and cygwin, I wanted to use : https://github.com/clj-python/libpython-clj

so my first step was : clone repo then : lein install

but this gives me an error: http://java.io.IOException: Cannot run program "clojure" (in directory...

@andreas.scheinert if you don't get answers here, there's a #libpython-clj -- also the data science folks are pretty active over at zulipchat

thank you sogaiu

for the time beeing , maybe I took a wrong step to begin with, I'm used to lein and project.clj files , but carin meiers examples use deps.edn

could you point me to a tutorial or something howi turn her deps.edn example project to a lein project.clj one?

this is her example project: https://github.com/gigasquid/libpython-clj-examples

@andreas.scheinert i don't have a tutorial like that on hand, but perhaps this sample from the libpython-clj creator that uses project.clj might be of some help? https://github.com/cnuernber/facial-rec

@sogaiu thank you I try this one!

thats so simple I wonder why its not included in the "official" example. It worked thanks @sogaiu!

https://clojure.atlassian.net/browse/CLJ-2056 (efficient alternative for (first (filter pred coll))) was rejected on the grounds that linear search leads to poor performance. However, I often find myself in a situation where I need to do it, especially when dealing with data from external sources. Is my best bet to just keep doing some variant of (first (filter pred coll)), or is there an alternative way to approach this problem?

I was looking at a jar file created using lein uberjar, and to my astonishment, it contains clj files as it is. How does java command handle it? Where can I find more information on it? Or am I looking at wrong place for .class files?

main idea is that in basic case clojure code compiiles on runtime

Nooooooo 😮

It means, compilation phase includes "reader" as well?

yes

TIL it. Thanks @nxtk!

check this talk https://www.youtube.com/watch?v=-Qm09YiUHTs

also it's good to know that clojure's minimal compilation unit is a top level s-expression\form, but there is some AOT functionality present like gen-class, usually for calling clojure from java.

I’m trying to follow this tutorial for using Docker and Postgresql but I’m getting this error when I follow the steps. https://circleci.com/blog/package-a-clojure-web-application-using-docker/

user=> (dev)
CompilerException java.lang.IllegalArgumentException: Must hint overloaded method: toArray, compiling:(clojure/core/rrb_vector/rrbt.clj:282:1)

Is (seq big-lazy-seq) lazy? We use this pattern often: (when (seq x) (process x)). We’ve been having Java Heap memory errors in a job that processes a big lazy sequence in batches (~1M records, 4 jobs in parrallel). Removing the (seq records) solves our problem. Memory usage would keep growing and GC would fallback on old-gen. These are our jvm opts:

-Xmx${max_heap_size} \
  -XX:+UseG1GC \
  -XX:MaxGCPauseMillis=100 \
  -XX:-OmitStackTraceInFastThrow \
  -XX:+HeapDumpOnOutOfMemoryError \
  -Duser.timezone=UTC \
  -Duser.country= \
  -Duser.language=en \
  -Djava.rmi.server.hostname=localhost \

@yannvahalewyn as far as i can see, seq is not lazy, it will realize LazySeq into a Cons seq

Object ls = sv;
		sv = null;
		while(ls instanceof LazySeq)
			{
			ls = ((LazySeq)ls).sval();
			}
		s = RT.seq(ls);
		}
	return s;

and each ls.sval() calls invoke() on a function wrapped by lazy-seq

Hmm, thanks for replying. I’m pretty confident seq will and should stay lazy though.

(defn my-inc [x] (println x) (inc x))
(take 10 (seq (map my-inc (range 200))))
;; prints out 0-31

take returns lazy seq

aha

map will be realized

let me see just to be sure

(do (seq (map my-inc (range 200))) nil)

Thanks @nxtk for thinking along! We have the same issue if we do (when (first records)). The thing that confuses me is that next we (partition-all 500 records) and that causes no issues whatshowever. This indicates there must be an issue upstream of some sort, but I’m not sure what I am looking for. Does anybody have some ideas what may cause this behavior?

too demonstrate

(defn my-inc [x] (println x) (inc x))
=> #'sicp.core/my-inc
(def xs (map my-inc (range 5)))
=> #'sicp.core/xs
(realized? xs)
=> false
(def ts2 (take 5 (seq xs)))
0
1
2
3
4
=> #'sicp.core/ts2
(realized? xs)
=> true

well it will realize up to 32 elements, for memory optimisations

my repl is broken 😅 , can you try with 100?

ok you are actually right ^_^

let's dig deeper then

i totally forgot about chunking

So we’re looking for something wrong that makes it so that using (for [batch (partition-all 500 x)] ...) won’t blow my laptop up, but (when (seq x) (long-process x)) and even (when (first x) (long-process x)) will :thinking_face:

where long-process is the batch loop being wrapped by the when clause

and x is a lazy seq which comes from a streamed json from s3. To be clear and avoid XY problems, all streaming works without issues without those when clauses. And what we need is to see if there is something to process, and that is essential (it’s a sync job, old data gets deleted and replaced).

hello! it seems to me that I have a problem with my clojure installation, currently I only have lein installed and when I try to run "lein repl" on this repo ( https://github.com/clj-python/clj-template) I get: http://java.io.IOException: Cannot run program "clojure" ...

is there a succinct way of writing assoc the k-v if k doesnot exist in the map?

if you don't care about the not-present/nil distinction then update

user=> (update {} :foo (fn [v] (or v 1)))
{:foo 1}
user=> (update {:foo 2} :foo (fn [v] (or v 1)))
{:foo 2}

otherwise no, you have to use contains?

Also, (fn [v] (or v 1)) is the same as (fnil identity 1). Assuming you don't care about false values.

Given a macro: user=> (defmacro m [x] #_=> `(prn ~@x))

(def a 3)

(m (list 1 a)) works. But

(def b [1 2]) (m b) failed.

why cannot the macro accept a symbol, but is allowed to accept a function result?

~@ can only act on a sequence, the symbol b is not a sequence (even if its runtime value is), '(list 1 a) is a sequence of 3 things, the symbol list 1 and the symbol a

thanks.

so it has nothing to the function list, it’s just ’() is the list.

yep, that's as I understand it

one thing I find quite helpful for understanding macros that you might not have seen is macroexpand, running for example: (note the backtick before the expression)

(macroexpand `(m (list 1 a)))

(clojure.core/prn clojure.core/list 1 user/a)

Is there a simple hello world example of using tools.deps with docker?

do you mean install it inside a docker container? specify "with"?

I can’t seem to run clj with this docker image. https://hub.docker.com/_/clojure

And there is no clear examples of getting up and going with docker that I can find.

I thought I would be able to get going quickly but seems I will need to learn docker before I can get going

It would be helpful to understand why do you want to run Clojure in docker - is for shipping production workloads? Working locally? By default clojure image ships with Leiningen, you can start it with docker --rm -it clojure lein repl

im using tools deps

I need to use docker because that’s how everything is deployed at work and they want it that way for consistency. I’m unfamiliar with this area as I am coming from front end.

That still doesn't answer the question - do you have some sort of backend written in clojure that you need to run locally via Docker (totally doable)? Do you want to work on a Clojure project and use Docker to develop a Clojure application (AFAIR not doable ergonomically)?

Ah sorry @lukaszkorecki I see what you mean. I have to develop against a Postgresql database locally and deploy it to AWS, likely several microservices all doing very much the same thing. The purpose is data preprocessing for data scientists.

@grounded_sage Why not develop in the most convenient way for you and then package the apps as uberjar files and run them inside Docker with java -jar ...?

Then you don't need anything inside the container except a JVM.

I’m unsure how all of this works so I can’t really comment. All I know is they want me using docker and pretty much every google search for docker and clojure results in. • Don’t use docker • We use docker but are not sharing how we did it - ( for whatever reason that may be, which is okay but somewhat frustrating compared to other lang ecosystems )

Usually Docker is used for packaging and deploying things, or running things like databases or already built services/apps

I'm not sure if you can develop in Docker, at least because of the file synchronization issues

Is there simple examples of a deploy?

That's a whole different can of worms, out of Clojure's scope really - do you have many hosts? one? Where are you deploying (aws? google?) - can you actually deploy etc etc etc

I mean there is this example. But it doesn’t run for me. https://circleci.com/blog/package-a-clojure-web-application-using-docker/

If you produce an uberjar from each of your Clojure apps, all you care about is "deploying Java apps with Docker"

Yea this seems like it’s going to be the best option. I think I just need to learn Docker more because trying to copy/adapt what I find online is simply not working

https://dzone.com/articles/run-simple-jar-application-in-docker-container-1 -- from the part about adding HelloWorld.jar to Docker

You'd want whatever base Java Docker image your company is using but it's effectively as simple as develop in Clojure locally, build an uberjar (using whatever build process you prefer: lein, boot, CLI/`deps.edn`) and then create a basic DockerFile that adds your JAR file to a Java-based docker image.

I run Clojure in ECS Fargate, and I don’t even bake my uberjar into the image. Instead I use two containers, one that downloads the jar from S3 into a place accessible to the other container. The other container is a plain openjdk container with zero modifications

Fargate calls these container dependencies, K8S calls it “init containers”

The jar pulling container only pulls the jar then dies with exit code 0

Whatever you do, just use java not lein/`boot` to start your app

my install method for java apps is similar, a freshly booted instance will figure out by itself what it needs to pull from s3 and run

and obviously asg iam roles guarantee access to the s3 objects accordingly per app

I do this because I have no need to add anything to the java docker image besides my jar, and complicating my CI with docker build/push isn’t worth it

A common way to develop services is to have postgres and other non-Clojure services running in separate docker containers, because that way you can quickly start them up without installing them and ensure you have the same version and similar configuration as when it is deployed, etc. However, the clojure service you are developing is not run inside a docker container because it’s much easier to start a REPL. When deployed, you might want to run your Clojure service inside of docker container to standardize and version the deployment environment, or use services like kubernetes / ECS / etc. It will typically talk to databases and other services that are outside of its container, though

Thanks @lilactown that’s a very clear response.

At work, I have Docker running MySQL (Percona 5.7), Elastic Search, and Redis -- docker-compose up i I need to get a dev env up and running -- then do all my Clojure dev directly via the REPL from my editor (including running tests etc), with occasional more complete test suite runs via the terminal/command-line. Then we build (uber) JAR files for each process and that's run via java -jar in production (we run on servers but it would be identical in Docker really) and all the database/search services are all running elsewhere in production (on a mixture of private and public cloud).

Our setup is somewhat similar: locally compose manages databases, rabbitmq etc and also can run our services streight from docker. In production we use managed services as much as possible, and deployments are just ansible runs pulling new versions of the container and restarting them. Our Dockerfile for Clojure uses multi stage builds with lein doing uberjarring and final container being just the JDK running the uberjar

@lukaszkorecki I’m curious as to what the multistage builds side of things are. Though it seems that most people using a docket file with build commands are using lein

https://github.com/kwladyka/fy-stock/blob/master/Dockerfile here is good example how you can use multi stages

other example I have not in public repositories

Can i see via REPL which protocols a Record is implementing ?

@matthewlisp (ancestors RecordName)

That will show all the types the record implements including protocols.

user=> (defprotocol Foo (bar [x]))
Foo
user=> (defrecord Quux [y] Foo (bar [_] y))
user.Quux
user=> (sort-by str (ancestors Quux))
(java.lang.Object clojure.lang.Associative clojure.lang.Counted clojure.lang.IHashEq clojure.lang.IKeywordLookup clojure.lang.ILookup clojure.lang.IMeta clojure.lang.IObj clojure.lang.IPersistentCollection clojure.lang.IPersistentMap clojure.lang.IRecord clojure.lang.Seqable .Serializable java.lang.Iterable java.util.Map user.Foo)
user=> 

@seancorfield ancestors is a superset of supers?

i think i wasn't clear about what i want

i want to know what protocols an INSTANCE of a record implements

if that's even possible

and if the output come along with this huge list, it won't be of much use to me 😞

(ancestors (type instance))

i wanted only what i've written

It's a set, you could easily remove the standard ones that all records implement.

amazing

@nxtk ancestors is a superset of supers, yes. I've never actually used supers -- I had to doc it in the REPL and then look at the source of ancestors to see it calls supers if it is passed a class instead of a tag .

ty Sean

user=> (defrecord base [])
user.base
user=> (def base-types (supers base))
#'user/base-types
user=> (apply disj (supers (type (->Quux 42))) base-types)
#{user.Foo}
user=> 

"It's just data" 🙂

WOW

so beautiful....

note this won't tell you if someone has installed a protocol externally for a record

and because that's an open set, there is no way to answer the general question (although you could answer it for specific protocols)

it's just for local protocols defined by myself

it's enough

True. Now you can extend protocols via metadata on objects, that changes the game a bit.

i'm making a little guide for my team-mate, we use Integrant lib at work

turns out we use Records + Protocols combo, and i was searching a way to dynamically find out which is the protocol that a Component in the system is implementing, so this way he can found it's "methods"

could be better if i could print out the name of the function(s) the protocol is specifying

but that's enough for now

protocols are maps that have a map of methods in them

user=> (defprotocol P (a [_]) (b [_]))
P
user=> (:sigs P)
{:a {:name a, :arglists ([_]), :doc nil}, :b {:name b, :arglists ([_]), :doc nil}}

Do protocols and spec play nice together? I have spec predicate

(partial satisfies? protocol/ToHoney)

java.lang.UnsupportedOperationException: Can't create empty: pyze.query_generator.pql.HoneyLiteral
	at pyze.query_generator.pql.HoneyLiteral.empty(pql.clj:70)
	at clojure.core$empty.invokeStatic(core.clj:5247)
	at clojure.core$empty.invoke(core.clj:5241)
	at clojure.spec.alpha$every_impl$cfns__2235$fn__2241.invoke(alpha.clj:1277)
	at clojure.spec.alpha$every_impl$reify__2254.conform_STAR_(alpha.clj:1291)
	at clojure.spec.alpha$or_spec_impl$fn__2115.invoke(alpha.clj:1089)
	at clojure.spec.alpha$or_spec_impl$reify__2118.conform_STAR_(alpha.clj:1100)
	at clojure.spec.alpha$or_spec_impl$fn__2111.invoke(alpha.clj:1065)
	at clojure.spec.alpha$or_spec_impl$reify__2118.conform_STAR_(alpha.clj:1100)
	at clojure.spec.alpha$conform.invokeStatic(alpha.clj:164)
	at clojure.spec.alpha$conform.invoke(alpha.clj:160)

Anyone knows what's up with core.typed? I thought I'd look what its state is, and to my surprise, the repo seems to get regular, frequent updates. But to my disappointment, the documentation hasn't been updated in years and is completely useless...

Ambrose just started working on it again recently and has been re-merging previously split libraries

I am not sure he's at a point where the docs can be regenerated yet

so I think you've just caught it at a particularly bad time

cc @ambrosebs in case he might see this

Ah, I see.