Fork me on GitHub
#tools-deps
<
2020-07-29
>
seancorfield01:07:01

(OK, running 1.10.1.596 on all our servers at work now 🙂 )

salam01:07:42

how would one represent this in deps.edn?

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>com.oracle.database.jdbc</groupId>
      <artifactId>ojdbc-bom</artifactId>
      <version>19.7.0.0</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependencies>
  <dependency>
    <groupId>com.oracle.database.jdbc</groupId>
    <artifactId>ojdbc10</artifactId>
   </dependency>
</dependencies>

salam01:07:37

if possible, of course.

seancorfield02:07:42

@abdusalam Pretty sure you'll need to figure out what that Bill Of Materials expands to and add each of those dependencies directly to deps.edn

salam02:07:25

thanks, Sean. does it mean there's no direct, built-in support in tools.deps for this kind of use case?

seancorfield02:07:55

Not as far as I'm aware.

Alex Miller (Clojure team)02:07:27

no, deps does not support the BOM / import scope stuff

Alex Miller (Clojure team)02:07:33

it seems radical, but deps just expects you to list the specific dependencies you're using :)

😂 3
salam03:07:00

i see. oracle's jdbc driver (com.oracle.database.jdbc/ojdbc10) used to list other "extra goodies" as dependencies in its pom.xml. i was looking for an elegant way to exclude the extra stuff in order to avoid doing the "exclusion dance" but just found out that it's dropped all of the dependencies since version 19.7.0.0. all we need to do now is to add com.oracle.database.jdbc/ojdbc10 {:mvn/version "19.7.0.0"} to get just the driver itself. 🙂

salam03:07:49

all good now

Alex Miller (Clojure team)03:07:20

yeah, that's all most people need. the whole bom thing seems like something an enterprise java developer thought up while they were high

😆 21
rickmoynihan06:07:50

So I really appreciate the expanded deps guide and reference, but there are two things from the previous iteration of the docs that I found useful that appear to have disappeared now: One was the old deps data-flow/arg/alias mapping diagram: https://github.com/clojure/clojure-site/blob/master/assets/images/content/guides/deps/deps.png which I appreciate might now either need to be two diagrams, or reworked into a more complex one. Whilst it’s not something I used often, I thought it was effective in the guide at showing what aliases influenced each phase of execution etc. And the other perhaps more importantly was the “Aliases” section in the old docs here: https://github.com/clojure/clojure-site/blob/285fda270445a70af9b1ec7bdb069309d2d224d7/content/reference/deps_and_cli.adoc#aliases which clearly described how keys in a deps file were used by the command line alias flags. Are there any plans to bring updated versions of these sections back?

Alex Miller (Clojure team)14:07:30

the docs are (always) a work in progress. I pulled the diagram b/c it was way out of date. I have updated it a number of times during the process of the basis changes but I have concerns about whether it actually provides that much value as it has grown increasingly complicated. All of the info from the old Aliases section is still in the doc.

👍 6
onetom19:07:14

i found that diagram quite helpful, when i was initially learning the clojure cli. in practice however, i never really had to use anything else then -A so far...

rickmoynihan07:07:02

Second question — how stable are the new features in tools.deps? In particular the -X feature and the args map data? Are they likely to experience breaking changes between now and the next stable release? I get the feeling not. Sometimes it’s not clear to me what is blessed as stable and isn’t… i.e. the installer, the command line args, or the tools.deps api. (Though I guess it’s all technically alpha because it’s t.d.a not t.d)

Alex Miller (Clojure team)13:07:08

should be stable unless we discover some reason to change. not expecting to discover anything but that's why we're letting people try it. I'm expecting the gap to stable to be like days, not a long time.

👍 3
seancorfield15:07:54

I was surprised that 1.10.1.590 seems to be the latest stable version on brew already, given the flurry of bug fix releases yesterday (1.10.1.596 and 1.10.1.600).

Alex Miller (Clojure team)16:07:25

So I’m curious why you said that

seancorfield16:07:33

I just did brew upgrade clojure on WSL1 on my Windows laptop last night and it updated me to 590. Hence my surprise.

seancorfield16:07:42

Note: brew on Linux.

Alex Miller (Clojure team)16:07:53

That’s homebrew core, which I no longer update

Alex Miller (Clojure team)16:07:38

clojure/tools/clojure is official

seancorfield16:07:53

I know. Hence my surprise 🙂

Alex Miller (Clojure team)16:07:12

Well I released 590 a week ago, just took me a while to doc and announce

Alex Miller (Clojure team)16:07:24

But I guess somebody updated it

thheller10:07:17

@alexmiller how come you are dumping all the new features into :aliases? kinda confusing to give aliases multiple meanings IMHO. maybe I'm just not seeing the bigger picture though.

thheller10:07:23

like why is the exec stuff in :aliases and not a new top-level :exec key or so?

borkdude11:07:46

as far as I have understood it: :aliases was always intended for naming a chunk of EDN data that could be re-used elsewhere, but it's only now that we can see that being used in more places

Alex Miller (Clojure team)13:07:37

correct - I would flip it around to say :aliases is a generic facility, and we are making more use of it

borkdude11:07:54

I wonder if you can combine -A and -X to include extra deps in an -X invocation

Alex Miller (Clojure team)13:07:44

that's kind of the idea

rickmoynihan13:07:23

:thinking_face: so you can change the function that executed with the default data (by changing the classpath via :extra-deps); but you can’t change the data for a function alias

Alex Miller (Clojure team)14:07:51

I'm not sure that's "changing the function" as much as supplying the function

Alex Miller (Clojure team)14:07:58

and you can override the data on the command line

Alex Miller (Clojure team)14:07:07

maybe I'm not getting what you're describing

rickmoynihan14:07:04

is that intended usage? It feels a bit weird to have the same fully qualified function name, but with two different definitions (in two different :paths for example). Would it not be more natural for -X :args to compose?

Alex Miller (Clojure team)14:07:25

I don't understand why you have that

Alex Miller (Clojure team)14:07:52

can you more fully describe what you're talking about, because I don't get it

vlaaad14:07:30

@rickmoynihan you mean something like that?

{:aliases {:deploy-to {:fn my.ns/deploy}}
           :prod {:args {:url ""}}
           :qa {:args {:url ""}}}
clj -X:deploy-to:prod

vlaaad14:07:34

I thought it would work, but it doesn’t..

Alex Miller (Clojure team)14:07:14

yeah, that's not a thing

Alex Miller (Clojure team)14:07:00

I would probably do this:

{:aliases
 {:deploy-prod
  {:fn my.ns/deploy
   :args {:url ""}}
  :deploy-qa
  {:fn my.ns/deploy
   :args {:url ""}}}

👍 3
Alex Miller (Clojure team)14:07:20

but there may be other variants depending on shared config, command-line overrides etc

Alex Miller (Clojure team)14:07:55

also my.ns/deploy could interpret data in :args as something to look up in :aliases via the basis

Alex Miller (Clojure team)14:07:14

you don't get that for free, but it's like 2 lines of code

vlaaad14:07:22

Yeah, I get what you mean. Basis FTW!

seancorfield15:07:17

The basis stuff looks very interesting, but you lose access to it if your build your artifacts as uberjars and run them via java -jar (although I guess you could still do -Dclojure.basis='{...}' 👀 )

seancorfield16:07:00

It almost makes me want to go back to running code in production via clojure instead 🙂

☝️ 3
rickmoynihan14:07:27

Yeah sorry… I think I misinterpreted the motivation behind what @borkdude was saying as a way of effectively doing what @dominicm was asking earlier about composing :args data. But I misread :extra-deps as being :deps. Either way it seems :deps would kind of allow the kind of thing @dominicm was asking for; but in a nasty way… i.e. you could swap the function rather than the data. e.g. - clj -A:myfn -X:args - clj -A:myfn2 -X:args Where myfn and myfn2 aliases defined the same ns but with different function definitions.

dominicm14:07:54

I want to run fn2 and fn1 in this example. Tbh, I assumed args and fn had to co exist.

Alex Miller (Clojure team)14:07:44

I'm unclear if you have a question, and if so, what it is...

dominicm14:07:10

Not a question, I already asked mine!

dominicm14:07:25

Just clarifying that I don't think this would allow for multiple functions like on after :)

rickmoynihan14:07:34

Not saying this is a good idea btw!!!

Alex Miller (Clojure team)14:07:06

one helper here is that :args can also refer to an alias instead of a map

rickmoynihan14:07:22

oh right… that is interesting!

rickmoynihan14:07:46

seems more useful than the :paths one

Alex Miller (Clojure team)14:07:34

well the :paths one is foreshadowing :)

Alex Miller (Clojure team)14:07:14

I did forget to mention that in my blog writeup, but I think it's in the docs

rickmoynihan14:07:19

I don’t recall seeing it

vlaaad14:07:28

yeah, me neither

Alex Miller (Clojure team)14:07:38

it was in some version of the docs but looks like it got lost in the N reworks I did

rickmoynihan14:07:45

😆 easily done

rickmoynihan14:07:23

wow — I was silently disappointed that something like this wasn’t there

rickmoynihan14:07:30

but it’s great that it is

rickmoynihan14:07:36

is there an example of how it works?

Alex Miller (Clojure team)14:07:44

{:aliases
 {:f1 {:fn my/fn1
       :args :data}
  :f2 {:fn my/fn2
       :args :data}
  :data {:something :here}}}
and then clj -X:f1 or clj -X:f2

vlaaad14:07:31

where else can I put aliases?!?!?!

Alex Miller (Clojure team)14:07:01

oh sorry, misread your question

Alex Miller (Clojure team)14:07:57

if you read https://insideclojure.org/2020/07/28/clj-exec/ check out the basis injection stuff there

dominicm14:07:41

Now we have runtime basis, does that mean you can go back and interpret the exec aliases however you like?

Alex Miller (Clojure team)14:07:11

you can write a program that uses alias data

Alex Miller (Clojure team)14:07:30

which is broader than what you said

seancorfield16:07:06

I asked in a thread, but I'll ask in the main channel for everyone to see: given that clojure.basis is a property added by the clojure command, what is the thinking behind leveraging this in production code if you normally build uberjars and run those with java -jar? All of that lovely combined deps.edn structure has gone by that point...

rickmoynihan09:07:14

I also had a similar thought when I stumbled across the basis property etc. Though I figured you could spit out the basis to a clojure.basis file inside the uberjar, and slurp it as an io/resource in your -main. However the big disadvantage to this is that the basis would be generated at build time rather than run time, so it would be more like a manifest, than what the basis is really supposed to be. On reflection I think your idea is better — though I’m not entirely sure what you’d use the basis for in that case, other than as a means of using the clojure tool to provide config to your app.

rickmoynihan09:07:22

I guess the only real advantage is using the clojure tool and a deps.edn to manage the classpath in production code, rather than the underlying java command line.

rickmoynihan09:07:40

I have in the past done things like java -cp:myapp-uberjar.jar:some/server/resources/* myapp.main to for example add assets to an apps resource path as a production overlay… so I guess using the clojure tool itself to help manage these kind of things with aliases might be useful; though I’m not convinced the benefits are huge

rickmoynihan09:07:32

though I guess it might sometimes be useful to for example provide profiles in production for enabling socket servers etc :thinking_face:

rickmoynihan09:07:38

Also another usecase is you could provide extra tooling for prod systems in this manner. e.g. a set of aliases for various production tasks, e.g. triggering a -A:backup alias over cron etc.

rickmoynihan09:07:42

and it would mean you don’t need to waste time inventing new command line parsers and config formats etc

seancorfield16:07:13

It almost feels like it's worth switching to using clojure to run uberjars via an alias in the deps.edn so that all that stuff remains available in a "production" style context...

ghadi16:07:24

my guess is that an uberjar tool could convey the basis into the jar

ghadi16:07:17

the more I think about uberjars, the more disadvantages I see vs. sync'ing deps

seancorfield16:07:34

The basis differs depending on the aliases used to run clojure tho', right?

seancorfield16:07:32

(because the paths and lib map etc can be different from invocation to invocation -- although the set of aliases etc remains the same)

ghadi16:07:44

I'm shooting from the hip

ghadi16:07:35

i'd mostly want the alias data

ghadi16:07:42

for stuff like config

seancorfield16:07:38

I'd add it to depstar but I think at this point I want to wait and see what tools.build ends up being 🙂

parrot 3
seancorfield16:07:08

I might experiment with using clojure to run uberjars tho' later today puts on mad scientist hat...

squirrel 3
ghadi16:07:27

using clojure instead of uberjars?

seancorfield16:07:31

Having an alias that treats the uberjar as a :local/root dependency and runs the expected -main function inside it, so you get easy access to the t.d.a. / Clojure CLI infrastructure

seancorfield16:07:43

This:

:aliases
 {:run       {:deps {worldsingles/worldsingles {:local/root "../../build/uberjars/worldsingles-1.0.0.jar"}}
              :fn worldsingles.publisher/-main}}
  ...
And then clojure -X:run

seancorfield16:07:38

So you get the "benefit" of an AOT'd uberjar with the benefits of the Clojure CLI. And the program can read the basis 🙂

seancorfield16:07:51

(of course, now you need the Clojure CLI installed on a target server, not just the JVM, and you need a deps.edn file in the folder where you start the app)

vlaaad16:07:15

And internet access

vlaaad16:07:25

And credentials for private repos

seancorfield16:07:32

@vlaaad Not if the entire :deps is just the uberjar, right?

seancorfield16:07:55

There are no other deps to fetch at that point.

vlaaad16:07:39

Ah, sorry, misread the situation

vlaaad16:07:05

I was thinking about using clj without uberjar

seancorfield16:07:41

Right, which we used to do on production (and, on one server, for "reasons" we still do that).

borkdude18:07:49

Not that I really care, just wanted to report that if you call a private function with -X, it also works:

[email protected] /tmp $ cat src/foo.clj
(ns foo)

(defn- foo [_]
  (prn :foo))

[email protected] /tmp $ cat deps.edn
{:aliases {:foo {:fn foo/foo}}}
[email protected] /tmp $ clojure -X:foo
:foo

😎 3
Alex Miller (Clojure team)18:07:06

that's not intentional but interesting :)

Alex Miller (Clojure team)18:07:21

it does make sense given the impl

seancorfield18:07:11

Because requiring-resolve can get at private Vars and there's no access check on that dynamic call path, right?

Alex Miller (Clojure team)18:07:56

right, just like calling through a var

borkdude21:07:31

The private var call still works with 619 🙂

Alex Miller (Clojure team)21:07:22

I don't consider it a bug

Alex Miller (Clojure team)21:07:37

private functions are only in your mind

borkdude21:07:26

I'm going to quote you on that when somebody will ask questions about this 😂

$ bb '(ns foo) (defn- foo [] :foo!) (ns bar (:require foo)) (foo/foo)'
:foo!

p14n18:07:40

ANybody else having issues with the recent tools-deps docker?

p14n18:07:12

Specifically

Caused by: Syntax error compiling . at (aleph/http/core.clj:251:3).
Caused by: java.lang.IllegalArgumentException: No matching method setTransferEncodingChunked found taking 2 args for class io.netty.handler.codec.http.HttpUtil

p14n18:07:35

(I'm making assumptions here because nothing else has changed)

seancorfield18:07:39

@p14n change in JDK version perhaps? Last night I was helping some beginners debugging a weird reflection-based problem that changed between JDK 11 and JDK 14 (it was broken on 8 and 11 but worked on 14).

p14n18:07:24

That's what I thought. I switched to openjdk-8-tools-deps (tools-deps now 11) but no dice yet

p14n19:07:48

Fixed by using clojure:openjdk-8-tools-deps-1.10.1.502 🤷

seancorfield19:07:32

A change in the default Clojure version perhaps, with stricter syntax checking?

ghadi19:07:52

i don't use any of the clojure:* images

ghadi19:07:09

they are not made by clojure team, and are not necessary IMHO

ghadi19:07:46

I use an adoptopenjdk image and then add the clojure installer

p14n19:07:12

Do you host your own?

cap10morgan19:07:27

How is the groupId derived in clojure -Spom? mine keeps getting reset to the artifactId when I run it.

seancorfield19:07:33

That's a bug. That should not happen.

seancorfield19:07:42

(and it's a new bug ^ @alexmiller)

cap10morgan19:07:27

ah, interesting. this is with 1.10.1.590.

Alex Miller (Clojure team)19:07:33

I don't think anything has changed there

seancorfield19:07:49

(! 1133)-> head pom.xml 
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="" xmlns:xsi="" xsi:schemaLocation=" ">
  <modelVersion>4.0.0</modelVersion>
  <groupId>seancorfield</groupId>
  <artifactId>next.jdbc</artifactId>
  <version>1.1.569</version>
  <name>next.jdbc</name>
  <description>The next generation of clojure.java.jdbc: a new low-level Clojure wrapper for JDBC-based access to databases.</description>
  <url></url>
  <licenses>

Wed Jul 29 12:17:19
(sean)-(jobs:0)-(/Developer/workspace/next.jdbc)
(! 1134)-> clojure -Spom

Wed Jul 29 12:17:30
(sean)-(jobs:0)-(/Developer/workspace/next.jdbc)
(! 1135)-> git diff
diff --git a/pom.xml b/pom.xml
index 3dce875..5632254 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1,7 +1,7 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <project xmlns="" xmlns:xsi="" xsi:schemaLocation=" ">
   <modelVersion>4.0.0</modelVersion>
-  <groupId>seancorfield</groupId>
+  <groupId>next.jdbc</groupId>
   <artifactId>next.jdbc</artifactId>
   <version>1.1.569</version>
   <name>next.jdbc</name>

seancorfield19:07:14

The only change it makes is to overwrite the groupId

Alex Miller (Clojure team)19:07:58

well, nothing intentional :)

😁 3
Alex Miller (Clojure team)19:07:27

can you file a jira, I'm juggling many conversations right now

cap10morgan19:07:30

@seancorfield The expected behavior was just that it should leave that value alone when the pom.xml already exists?

souenzzo19:07:02

The function that turns (merge-cli-ops {:default :opts} "[:extra :opts]" "true") => {:extra {:opts true} :default :opts} is public?

Alex Miller (Clojure team)19:07:45

sorry, I don't understand the question

Alex Miller (Clojure team)19:07:09

or how it relates to tools.deps

souenzzo19:07:19

I liked the "DSL" to "assoc-in" values.

Alex Miller (Clojure team)20:07:02

Oh, yes that’s it. Treat anything about this code as implementation detail though

Alex Miller (Clojure team)20:07:27

It may migrate into Clojure itself

👍 3
seancorfield19:07:03

@cap10morgan Yup, it should only touch the dependencies in an existing pom.xml file.

👍 3
Alex Miller (Clojure team)19:07:20

well, it's broader than that - deps, repos, directories

Alex Miller (Clojure team)19:07:00

there has been some work done internally here to make this more configurable and that will eventually make its way into user configurability

3