if I have [1 2 [3 4 [5 6 7]]], can I get a pointer to [5 6 7]?

yes, but the answer depends on what rule leads you to it, and what you mean by "pointer"

(peek (peek v)) returns that specific immutable vector, for example, and in java all first class non primitive values are pointers into the heap

but you might want an actual pointer reference you could pass to a C program by address?

(that's possible, depends on which lib you use for ffi)

@noisesmith I have a tree with each node keyed like [:node1 "foo" [:child1 "bar" [:child1-1 "a"] [:child1-2 "b"]] [:child2 "baz"]] and I want to return a node and its children given a key.

The plan was to have a hashmap with the node keys paired to node pointers for near constant speed access.

a binding is a pointer, you can reduce over the tree-seq of the collection then call group-by

that gives close-enough-to-constant-time access via binding each key in the map to the corresponding subtree

@deadghost

user=> (->> [:node1 "foo" [:child1 "bar" [:child1-1 "a"] [:child1-2 "b"]] [:child2 "baz"]] (tree-seq coll? seq) (filter coll?) (group-by second) pprint)
{"foo"
 [[:node1
   "foo"
   [:child1 "bar" [:child1-1 "a"] [:child1-2 "b"]]
   [:child2 "baz"]]],
 "bar" [[:child1 "bar" [:child1-1 "a"] [:child1-2 "b"]]],
 "a" [[:child1-1 "a"]],
 "b" [[:child1-2 "b"]],
 "baz" [[:child2 "baz"]]}
nil

due to the mechanics of group-by each val has an extra vector around it (in case the same key came up twice)

it's straightforward to write a "key-by" that works like group-by without the vector

looks good speedwise

especially if you do multiple lookups it should be good

looks like it can consume quite a bit of space if the tree is deep

it uses only one kv pair worth of memory per entry beyond your input

it's not copying the data, just indexing into it

that's safe, since the input is immutable

hmm ok that's perfect then

I'll digest it a bit

thanks

@deadghost possible definition of "key-by" plus the output when using it instead of group-by

user=> (defn key-by [f coll] (into {} (map (juxt f identity)) coll))
#'user/key-by
user=> (->> [:node1 "foo" [:child1 "bar" [:child1-1 "a"] [:child1-2 "b"]] [:child2 "baz"]] (tree-seq coll? seq) (filter coll?) (key-by second) pprint)
{"foo"
 [:node1
  "foo"
  [:child1 "bar" [:child1-1 "a"] [:child1-2 "b"]]
  [:child2 "baz"]],
 "bar" [:child1 "bar" [:child1-1 "a"] [:child1-2 "b"]],
 "a" [:child1-1 "a"],
 "b" [:child1-2 "b"],
 "baz" [:child2 "baz"]}
nil

dumb question: is clojure.lang.Box a thing one could use or is it an implementation detail?

First time I've ever heard of it so I'd say "implementation detail".

yeah i first heard of it 5 minutes ago in this stack overflow post

https://stackoverflow.com/questions/31288608/what-is-clojure-volatile

the top answer quotes some source that mentions it

It's a weird class. A single public member val of type Object 👀

all of the clojure implementation feels like it was written in a way to intentionally make people who write java mad

Hahaha... yeah, almost all Java devs hate Rich's indentation/brace style in his Java code too 🙂

i hate the huge blocks of commented out code

I can see a parallel between that and Rich Comment Forms... Java doesn't really have a way to say "Here's some code that I want to use for testing/benchmarking things but I don't want it in the main compiled code"...

It's not worth the effort of trying to shoehorn that into some sort of standalone test code, given the way Java build tools work, so I can understand this as a compromise.

(hate was too strong of a word - more a strong eh)

found an example not related to test code

Given the evolution of Clojure, I actually find that useful: it shows how the code was and what it is now, at a glance. That heads off people who think they've found a bug and go "Hey! prepRet should probably check Integer and Float right?"

Since now they'll look at it and go "Oh! It used to check those, so there's got to be a good reason why it doesn't now"

(mind you, commenting out vs removing isn't consistent)

And, to be fair, even the Clojure code in the compiler/runtime isn't idiomatic a lot of the time -- and that's almost always true of compiler/runtime system code. I've worked with a lot of compilers over the years and you often have to do some serious contortions to produce the right result in a performant manner.

Back in the 80's I wrote large parts of a C compiler and runtime, based on a VM architecture, and we chose to avoid all implementation-defined, undefined, and unspecified behavior in the system itself -- so you end up writing a lot of peculiar C code for that.

I've also worked on a COBOL compiler (written in COBOL!) and several other compilers/interpreters/etc. So much non-idiomatic code.

Package and tool maintainers: Do you pre-compile your .class files and bundle them into the Clojars release? What are the tradeoffs here, I feel like you could run into JVM version issues on user's machines?

does anyone have any tips/resources/libraries/examples for mechanically generating Clojure wrappers for large Java APIs?

clojure spec uses a global registry of keys. https://github.com/weavejester/integrant also uses a global registry of keys. (There's also spec coerce and a few other things that lean on spec). Are there any other examples of libraries or open source apps that use a registry of keys with functionality assigned to each key?

namespaces and vars

true but I'm thinking more about things that implement functionality for "nouns" at the application level

spec lets you to assign specs to these "things" and then you get a set of operations for those things. Integrant let's you start with data associated the things then implement multimethods for them

it's not different

they are both global addressing schemes for locating something

yeah i'm starting to think about if there's a way to separate the registry itself from the operations

like so you could just have a big map atom of the globals keys then each value would be a map that could look like

{:spec int? :ig-init #(.start %) :render render-fn} 

why not just use a multimethod or protocol in that case?

a protocol is actually implemented as a map of functions

hmmm yeah that might work. I was sort of thinking that it also might be useful to store data here and not just functions, and to have this be enumerable. This might be mixing up too much stuff though

there are cases where doing stuff like this is useful, but if you can mix the stuff already in the box, it's likely to be less fragile over time

protocols have a lot of special advantages (perf, callsite caching, integration with java, inline implementation in records+types+reify, etc) so think carefully about the tradeoffs

by protocol you mean using a record type to implement the protocol on instead of using global keys right?

So you'd need a record/type for each noun

For thoose of you who are doing "non-blocking" coding with e.g core.async or manifold. How do you tame jdbc? Do you hide things behind e.g an ExecutorService?

jdbc is always synchronous, so yeah your only option is to submit it to a threadpool and return futures/channels

@jarvinenemil by not using it 😬

if we had to do some jdbc stuff we would use a manifold future, which uses an ExecutorService behind the scenes and presents the same promise-like interface of manifold deferred

You can use core.async/thread

it is likely that a future JVM that includes Fibers will not have to do anything special to be non-blocking, even with JDBC

it will be possible to have the semantics of <! >! within an ordinary thread, no need for go blocks

> it is likely that a future JVM that includes Fibers will not have to do anything special to be non-blocking, even with JDBC Project Loom can’t come soon enough 🙏

How can I use rlwrap for sane arrow behavior but also use jline for reading a single character in Java?

it seems once you use jline to read a single char, rlwrap behavior stops working

can't jline also give you arrow behavior?

maybe, but I don't know how it works. I just googled around for how you read a single char in clojure and then I found that

I think on a TTY level, once you want character as opposed to line oriented actions it's one or the other

right, maybe I shouldn't go down this rabbit hole...

they both want to control the terminal IO

this lib copies some stuff from Repl-y https://github.com/omnyway-labs/clj-jline/blob/master/src/clj_jline/reader/simple_jline.clj

also line-based right?

any reason tools.deps CLI is using rlwrap vs this?

but it has jline interop so you should be able to get character input too once it's set up, one would hope

I'm not sure. Perhaps it's that rlwrap is a simpler solution out of the box, and pulls in less libs.

going with rlwrap for now and just lines... this is for another time. thanks

there's also lanterna for more detailed terminal control

I stumbled on that, but the source code seems to have disappeared

https://multimud.github.io/clojure-lanterna/

ah thanks, I got a couple of dead links to bitbucket 😄

but if using jline for line+characters is already out of scope, lanterna is even further left of field

yeah, I'm going to figure this out for a next iteration

(.read *in*)

I think rlwrap even interferes with that

user=> (char (.read *in*))
A
\A
user=>

does not

it interferes by requiring a newline

it's not a single character consumed

doesn't work on my machine, fwiw

what does "doesn't work" mean?

you want to read a single character without enter, as @noisesmith suggests, or some other error?

the most common way to break that is running in nrepl

without enter, correct

you need true terminal controls to get that unbuffered access

(aka jline or lanterna or the like)

I just wrote a tool for that

https://github.com/ahungry/rawterm

@ahiguera coll - is it cross platform? (noticing it's a C lib...)

oh, docker :D

How can I remove a map from a vector? Let's say I want to remove the map with :id 2 and I have:

(def fruits [{:id 1 :name "apple"} {:id 2 :name "orange"} {:id 3 :name "water melon"}])

you can't remove things from the middle of a vector (efficiently)

why not a map keyed by id?

it cool be nice yeah, but I was lazy to figure how update my forloop 😅

Should I use doseq and map keyed by id as you suggest?

(def fruits [{:id 1 :name "apple"} {:id 2 :name "orange"} {:id 3 :name "water melon"}])
=> #'user/fruits
(def indexed-fruits (into {} (map (juxt :id identity)) 
                          fruits))
=> #'user/indexed-fruits
indexed-fruits
=> {1 {:id 1, :name "apple"}, 2 {:id 2, :name "orange"}, 3 {:id 3, :name "water melon"}}
(dissoc indexed-fruits 2)
=> {1 {:id 1, :name "apple"}, 3 {:id 3, :name "water melon"}}
;; lazy
(remove (comp #{2} :id) fruits)
=> ({:id 1, :name "apple"} {:id 3, :name "water melon"})

I think I'm going with

(def fruits {:1 "apple" :2 "orange" :3 "water melon"})
(for [[k v] fruits]
...)

map "keys" can be any clojure "data" (aka you can do {1 "apple" ...}

is there a way to signify that java sources should be compiled with deps.edn?

no

deps.edn isn't in that business

also, what are the "rules" for when i point deps.edn to a library on git

if i just had a java project with a pom.xml would that work?

what about a leiningen project?

or does it have to be a pure clojure, deps.edn specified thing

I have used deps.edn to specify deps that are Maven-central published Java libraries just fine.

They had no deps.edn file of their own, but presumably must have a pom.xml

I believe (Alex will chime in with the gospel) that clj will read transitive deps from pom.xml and project.clj -- but note, if the project needs javac compilation (likely with pom.xml) it will not do that for you

a good rule of thumb is you can only use other deps.edn libraries as git dependencies, it is a little more nuanced the that, it does support some other formats

but there are more corner cases in the other formats

it definitely doesn't support anything that requires a build step as a git dependnecy

so java source is right out

thats...somewhat frustrating

but okay

I still don't fully understand the point of deps.edn

like leiningen and maven cover |------------------| these use cases

deps.edn covers |------| these

but i dont know what to use for the remaining |----------|

in theory i like that it is a more focused and narrow tool

@emccue I think this is / is meant to be / will be the other remaining part https://github.com/clojure/tools.deps.alpha/wiki/Tools

I haven't tried it, but this might be an option for compiling a java project for deps.edn https://github.com/danielsz/meyvn

using mvn to compile the java looks like a good choice to me

hmm - maybe not, its docs don't actually mention using javac in mixed clojure/java repos

there are tools that add a build step on top of deps.edn, but I think most of those tools are aimed at building your own java source, not building the source of dependencies, but maybe there is one somewhere that does that as well

oh yeah, that is a distinct issue - it seems like the java ecosystem isn't really set up for source deps

its somewhat annoying because deps.edn is the only thing that works with git deps directly afaik

so if i put a library on a private git and that is my method of deployment i am locked into only using it with pure clojure projects

because i can only pull it with another deps.edn

you can build it and deploy it into a maven repo (public or private) and use it from any build tool

yeah that seems to be the way to go in general

clojure has the advantage of being a language that can work from source. git deps let you take advantage of that, when possible.

when not possible, fall back into the "java way"

what options are there for private maven repos these days?

ive used artifactory before but only as a consumer

i have no clue what it takes to set up something like that in a reliable way

you can just stick stuff in an s3 bucket if your needs are pretty minimal

you can mvn install directly to local cache if you aren't depending on an external CI

that s3 bucket option sounds interesting

yeah, the s3 mvn thing works mostly

https://github.com/s3-wagon-private/s3-wagon-private

deps / clj are set up to work from s3 repos

so you don't need to add anything additional to do that

incl private s3 repos

it uses the s3-wagon-private thing but fixes several obnoxious things about it (which yes, I've filed upstream)

what are the cons of using s3?

http://deps.co is great

I know with terraform there is a whole system with a locking table w/ dynamo to maintain atomicity

so i would anticipate race-condition flavored concerns

I don't think people use/think about maven repos in that way

this is simple put into an s3 bucket

the s3 wagon works great

well, I wouldn't say "great", but it does kind of work

good to hear there are fixes coming to s3 private wagon

it relies on an ancient version of the amazon stuff and the way it expects creds is totally broken - there's an issue there with all the details. I hacked over it for the deps usage of it

we use it at work, but at random times it reads metadata during our build process, even when just recompiling CLJS. might also be a boot thing, haven't figured it out

oh yeah... I forgot the creds stuff

ptsd

I've actually rewritten the whole mess (mostly) using the cognitect aws-api client, just haven't gotten around to finishing it, but will swap that whole thing out at some point

would be cool to have that as a drop-in replacement

it's not a wagon, I'm actually using the newer maven apis directly

so it won't be useful for mvn / lein afaik

Also last I checked there's the cosmetic issue that s3-wagon-private prints stack trace logs every time you check it for a dep that isn't in that repo

so you can override repo search order to make sure it's checked last, or sort through lots of spam

a very neat project for someone looking for something to do would would be a way to run maven repos p2p ala syncthing or ipfs

the usage side and connection to tools.deps for something like that is pretty easy

it depends whether you want to play the maven game or the tools.deps procurer game

if you just want to run <handwave> some kind of storage </handwave>, tools.deps doesn't require much

I've looked at npm for example, and I think it would be trivial to hook that into the resolver. what you actually do with the result of that re building classpaths, I'm not so sure

(import 'leftpad) of course

if it would be useful to cljs to somehow have npm modules (w/transitive deps), I think that's easily within reach, but that's a solution in search of a problem

I think a lot of node modules surprisingly have a build step

wouldn't surprise me

I'm not saying this is good or useful, just an example that transitive resolution only has like 2 or 3 operations to implement