Fork me on GitHub
#beginners
<
2020-05-24
>
factorhengineering00:05:22

2. Beyond installing Node.JS (and NPM), the guide suggests just running the following to get shadow.cljs:

npx create-cljs-project my-project
and I did that. I do now see "For convenience you can run npm install -g shadow-cljs or yarn global add shadow-cljs. This will let you run the shadow-cljs command directly later. There should always be a shadow-cljs version installed in your project, the global install is optional." Is that a good idea?

jayzawrotny00:05:25

I don't think that's really necessary for now

factorhengineering00:05:24

A bit farther down I see "If https://shadow-cljs.github.io/docs/UsersGuide.html#_installation globally, you can use the shadow-cljs command directly.

$ shadow-cljs help
" and "The guide will assume there is a global install to keep examples short but this is not required."

factorhengineering00:05:38

So...it really seems like I should do that global installation step.

jayzawrotny00:05:46

If you don't go global, you can just use npx shadow-cljs help

qmstuart00:05:37

I had the same confusion in that guide

qmstuart00:05:51

What are the advantages / disadvantages of npx v global?

qmstuart00:05:02

I was confused as I'd never used npm or NodeJS before

alidcastano00:05:09

I usually install it as a dev dependency in any project

jayzawrotny00:05:30

Well the big disadvantage of global is that if you work in various projects, they may rely on a specific version of shadow-cljs

factorhengineering00:05:16

I do see the suggestion for npx shadow-cljs help. but if I run that I get:....

factorhengineering00:05:37

npm ERR! code E404 npm ERR! 404 Not Found - GET https://registry.npmjs.org/shadow.cljs - Not found npm ERR! 404 npm ERR! 404 '[email protected]' is not in the npm registry. npm ERR! 404 You should bug the author to publish it (or use the name yourself!) npm ERR! 404 npm ERR! 404 Note that you can also install from a npm ERR! 404 tarball, folder, http url, or git url. npm ERR! A complete log of this run can be found in: npm ERR! C:\Users\Ady\AppData\Roaming\npm-cache\logs\2020-05-23T2358_07_860Z-debug.log Install for [ '[email protected]' ] failed with code 1

jayzawrotny00:05:39

I think you typed shadow.cljs not shadow-cljs

factorhengineering00:05:07

but still...the entire guide is written assuming the 'global installation', so...sounds like I should.

jayzawrotny00:05:29

It's up to you, if that would make it easier to follow along it's worth it.

didibus00:05:55

Global is more convenient, and for a tool you want to install, I feel it makes sense. You can re-install things as local later if you need a specific version for a specific project I believe

factorhengineering00:05:43

Makes sense. I will. I had a 3rd question...now what was it?...

factorhengineering00:05:26

3. The http://clojurescript.org quick start guide reads that you need Java 8 and teh cljs.jar file

factorhengineering00:05:10

I asked here where I supposed to put the cljs.jar file, and someone told me I actually shouldn't download it.

factorhengineering00:05:17

So I skipped that and by the time I had installed Node.JS (and NPM) and shadow.cljs (by creating a project, so not actually installed), I kind of figured the cljs.jar file would have appeared somewhere by now.

factorhengineering00:05:37

I did a search in the project folder, though, and it's not there. So...what am I supposed to do? Just ignore the fact that I do not have what the main clojurescript website tells me I need? ...or just download it anyway, and if so put it where?

jayzawrotny00:05:07

Wait, are you following both the http://clojurescript.org tutorial and the shadow-cljs tutorial?

factorhengineering00:05:54

I was going to follow the http://clojurescript.org quick start guide (which I did last year on a different computer) just to refresh my memory about project folder set-up and compiling etc.,...until I was convinced by people here just to use shadow.cljs

factorhengineering00:05:10

I had barely started into the http://clojurescript.org guide, just installing JDK 8 (actually adoptOpenJDK 8 ). Did I not even need to do that??

jayzawrotny00:05:36

JDK 8+ is required for both shadow and the http://clojurescript.org tutorial. However, the http://clojurescript.org tutorial is only the minimal working setup. Meaning you wont be able to connect cider (or most editors) to the REPL out of the box.

jayzawrotny00:05:32

Shadow-cljs is nice because it can grow with you. The defaults are good for getting your feet wet, but if you decide to use it for more serious projects you'll be able to work with it and reap benefits of what it can do. The http://clojurescript.org tutorial may get to the point of writing cljs -> js but you'll likely soon outgrow it.

seancorfield00:05:32

I'd love to know how Cognitect folks build cljs projects -- what tooling they use?

jayzawrotny00:05:57

That's a good question. I can't imagine working without the hot-reloading the other tools offer.

didibus00:05:04

I suspect they use figwheel-main.

didibus00:05:20

But I'm doubting they use shadow-cljs

seancorfield00:05:25

Several of the high profile Cognitect folks have talked about their simple tooling workflow, which is why I wonder about their cljs workflow...

seancorfield00:05:59

...I think I have a reasonable understanding of how they approach Clojure tooling 🙂

alexmiller01:05:38

I think it varies a lot depending on the project

seancorfield01:05:00

I think Cognitect folks have talked quite a bit about Clojure tooling -- but very little about ClojureScript tooling, at least for the last several years.

seancorfield01:05:26

So what I mean is that I literally have no idea how y'all build ClojureScript stuff 🙂

alexmiller01:05:40

And we work on both greenfield and existing projects so it’s not always “our” workflow

didibus01:05:30

David Nolen no longer works (never has?) at Cognitect. I'd be curious what his tooling of choice is.

alexmiller01:05:36

I personally don’t know the answer to that, not really my area :)

didibus01:05:46

Hum, no wait I'm getting confused with Clojure CLR ok no that's correct.

seancorfield01:05:00

Good point re: greenfield vs existing projects.

alexmiller01:05:13

David did work at Cognitect for a number of years

didibus01:05:15

Ah okay, ya that's what I felt I remembered. Its awesome that he's continuing all his hard work even now that he doesn't. Feel very blessed to have all the core contributors put in all this work so I get to have more fun programming 🙏

didibus00:05:18

I think the getting started guide might be outdated for windows. Since then, both lein and tools.deps can run on windows and pull in ClojureScript for you, and I'd assume shadow as well

seancorfield01:05:26

@didibus There's no official Windows solution, hence the official docs must still suggest a more manual approach.

seancorfield01:05:34

(for what it is worth, I've run the cljs quick start guide on Windows Powershell using the CLI tools in their alpha state and it is possible)

jayzawrotny01:05:35

Hmm I do have access to a virtual windows machine I use for gaming. I'm going to try installing shadow-cljs on it and see what I run into.

didibus01:05:50

Ya that's fair. I guess lein doesn't count as official, and tools.deps powershell is alpha. My favourite on windows by the way is using the bb port https://github.com/borkdude/deps.clj

factorhengineering02:05:37

@jayzawrotny @didibus @seancorfield Sorry. got sidetracked there. Eccentric J, good to know that I shouldn't try to do both. I actually hadn't decided what I was going to do next yet. Still slogging through set-up it seems. Good to know there's one thing (the http://clojurescript.org tutorial) I can take off my list, and I should just keep going down the shadow.cljs tutorial page. Didibus, if I remember correctly the http://clojurescript.org quick start guide uses Leiningen.

dpsutton02:05:09

It does not use lein

factorhengineering02:05:16

So can anyone explain the apparent lack of a cljs.jar file on my system (or where it might be), and what I need to do about it, if anything?

factorhengineering02:05:39

It doesn't use lein? Hmm. Well it was about a year ago I did it, so I guess after I did it I must have tried out Leiningen for some reason.

alexmiller02:05:33

Both lein and deps will cache jars under ~/.m2/repository

factorhengineering02:05:33

I'm not using lein or deps. People recommended I use shadow.cljs instead.

factorhengineering02:05:49

(for CLJS, which I'm going to try for a while before I try Clojure)

factorhengineering02:05:28

Should I just not use the shadow.cljs guide if I'm on Windows, by the way? It seems to assume Linux.

factorhengineering02:05:44

I just tried to do the global installation as specified, and I get this:

factorhengineering02:05:19

npm install -g shadow.cljs npm ERR! code E404 npm ERR! 404 Not Found - GET https://registry.npmjs.org/shadow.cljs - Not found npm ERR! 404 npm ERR! 404 '[email protected]' is not in the npm registry. npm ERR! 404 You should bug the author to publish it (or use the name yourself!) npm ERR! 404 npm ERR! 404 Note that you can also install from a npm ERR! 404 tarball, folder, http url, or git url. npm ERR! A complete log of this run can be found in: npm ERR! C:\Users\[...]\AppData\Roaming\npm-cache\logs\2020-05-24T0228_55_872Z-debug.log

factorhengineering02:05:12

Actually if I go to the website https://registry.npmjs.org/shadow.cljs I get the same error.

dpsutton02:05:12

It was pointed out earlier that it is shadow (dash) cljs not shadow (dot) cljs

factorhengineering02:05:58

I could have sworn I copied and pasted that. My mistake! Thanks.

dpsutton02:05:00

no worries. if you want to see a presentation about how to use shadow-cljs, a sample app, and that app deployed on netlify check out https://github.com/dpsutton/asg-ignite-presentation

jayzawrotny02:05:36

That's great!

factorhengineering02:05:14

Thanks. Reading it now.

factorhengineering02:05:37

Still wondering about that maybe missing cljs.jar file, by the way...

factorhengineering02:05:54

Shouldn't it have installed with shadow?

factorhengineering02:05:34

I'm just going to run a search on my whole drive in the meantime.

jayzawrotny02:05:36

shadow-cljs should handle that for you, the jar is not necessary. You should only be using the shadow-cljs CLI.

factorhengineering02:05:01

Did I also not need to install Java 8 then?

jayzawrotny02:05:08

You do need Java 8

factorhengineering02:05:40

By the way, does it really have to be specifically 8, or can it be higher (I think 14 was suggested on a couple of JDK sites).

dpsutton02:05:58

8 or 11 are recommended. others most probably work but how many battles do you want to fight

dpsutton02:05:20

i think i'm using 13 with no issues.

jayzawrotny02:05:35

11 does has a few small incompatabilities with various tools here and there and if you already have 8 installed I would just stick with it for now.

factorhengineering02:05:50

Oh I don't want to fight any extra battles. lol I just can't remember where I got the impression that the version number even mattered. Maybe a guide meantioned 8 but was written when 8 just happened to be the newest.

factorhengineering02:05:04

8 all the way. thanks.

seancorfield02:05:38

You need at least version 8, that's all.

dpsutton02:05:29

i thought version 10 never really worked for some reason? don't really know but i think most people recommend the long term versions which are 8 and 11

seancorfield02:05:52

8, 11, 14 are safe choices for most Clojure stuff.

seancorfield02:05:25

The inbetween versions are very short-lived now, the way Oracle is developing Java/JVM stuff so I personally would avoid them (and recommend beginners avoid them, in case they do cause problems).

seancorfield02:05:08

We still have a few things on 8 but mostly we run everything on 11, although I'm developing on 14 now, mostly as a compatibility test for rolling all our systems up to 14 at some point.

factorhengineering02:05:12

Is there some kind of security argument for going with 14 instead of 8?

factorhengineering02:05:37

alrighty then. 8 it is

seancorfield02:05:56

At least, none that I know of. 8 is still actively supported by several companies and organizations. e.g., AdoptOpenJDK.

factorhengineering02:05:06

Ah...so 8 gets updated with lower-order version increments?

seancorfield02:05:41

https://adoptopenjdk.net/ shows update 252 is the latest version of 8 they're offering.

factorhengineering02:05:31

OK. so lower-numbered versions are not necessarily older. Got it.

seancorfield02:05:08

Well, JDK 8 as a whole is older than 9, 10, 11, 12, 13, 14 🙂 But each major version continues to get minor updates for security and buf fixes.

factorhengineering02:05:50

@dpsutton I'm reading through your guide/presentation, and...stop me if your intended audience isn't people who have only done a thing or two in JS and CLJS, but...I am a bit unclear about a couple of things already:

factorhengineering02:05:03

1. Your very first suggested command is 'mkdir app', but I have no clue what folder that should be made in.

dpsutton02:05:45

anywhere. you are making a new directory to work in. it doesn't make any assumptions about where it is

dpsutton02:05:19

it can be in whatever temp directory you like or in your projects folder, or a clojure directory, or in My Documents or whatever. you are making the directory

dpsutton02:05:46

the point of using mkdir is to show that it shows you everything you need. the directory is literally empty except for what we are about to put into it

factorhengineering03:05:43

Is it advisable to be somewhere in your HOME folder (I just created that for Doom Emacs installation, but I also started the first couple of steps of the shadow.cljs website guide there too, creating a project there.

jayzawrotny03:05:08

It's a common practice but could be anywhere.

dpsutton03:05:37

i don't believe there is any advice to give here. make a new folder wherever you think it should go

factorhengineering03:05:27

OK...so anywhere. I guess I would just make it in the same parent folder as where the top folder created by the shadow.cljs website guide project creation command gets put.

dpsutton03:05:50

is that another project?

factorhengineering03:05:57

Looking ahead to the next steps, of creating a few files...

factorhengineering03:05:11

2. There is quite a bit of (boilerplate) text to be put in the file, but no explanations of the text. Some of it (e.g. the html) I can follow, but it would be nice to know what some of it means.

factorhengineering03:05:22

More importantly, though...

dpsutton03:05:59

(this is a lightning talk to get you up and running in 5 minutes. everything should seem sensible and is explained in the users guide)

factorhengineering03:05:51

3. After the "Create shadow-cljs.edn" step" there is a "Index.html" step. Should that be "Create Index.html"? ...and according to the little tree diagram should that be in a subfolder of the 'app' folder called 'public'?

factorhengineering03:05:39

If you're open to suggestions (on how to get through to people at my level :P), I would change the titles for the files to be created to all be "Create <filename.ext> in:

factorhengineering03:05:45

" with a tree below.

factorhengineering03:05:27

Would be clearer. Only saying that because i noticed you requested suggestions. 🙂

dpsutton03:05:05

this presentation was originally given accompanying a talk where i could flush out details. owing to this i had to keep the slides simple and visible from a distance so lots of context was supplied from speaking

factorhengineering03:05:32

Sure, but the amount of context in the actual text of the presentation seems to change over the course of the list. 😉

factorhengineering03:05:13

2nd file no 'create'. 3rd file also no tree. 4th file...actually I can't really tell if there's a 4th file. Does the 'Closer look' section mean 'Create main.cljs in \app\src\app\'?

dpsutton03:05:42

what is the third file with no tree?

dpsutton03:05:59

closer look and boiler plate are both the same file

dpsutton03:05:05

just had to break it into two slides

dpsutton03:05:52

you can reference the repo for checks of what is going on

factorhengineering03:05:45

The 3rd file with no tree was the Hiccup section. I just saw your answer in the other thead, about how that section is not about a file to creat at all.

factorhengineering03:05:58

"closer look and boiler plate are both the same file" Ahhh. OK. and now I see it's all together in the Source files section.

factorhengineering03:05:12

...but that section is just one source file, yes?

dpsutton03:05:32

yes. i'll link it again here so we can stay in a single thread

factorhengineering03:05:37

Oh wow. There's a lot more in that main.cljs file than the presentation/guide contains. OK.

dpsutton03:05:23

its on slide 16

factorhengineering03:05:36

So...running 'npm i react-modal' makes the middle part of the main.cljs file appear??

dpsutton03:05:12

no. npm i react-modal installs the javascript dependency. then you edit the file to use it as shown on slide 16

dpsutton03:05:43

i'm putting the new source and leaving to you to see that ["react-modal" :as Modal] is new along with the other lines

factorhengineering03:05:42

I think it's coming together for me.. It's a bit confusing that you repeat some but not all of the previous contents of the main.cljs file.

dpsutton03:05:09

Yes I’m trying to focus on the diff

dpsutton03:05:25

But go see the complete file on github to see what it all should look like

factorhengineering03:05:46

I think if you included the whole thing on that step of the guide (that slide), including the part at the bottom that you refer to as 'boiler plate' earlier, it would be a lot easier to see what's going on.

factorhengineering03:05:35

Or at least replace the boilerplate part and the beginning part with [...]

factorhengineering03:05:43

Beginning part:

(ns app.main
  (:require [reagent.core :as r]
            ["react-modal" :as Modal]))

(.setAppElement Modal "#app")

factorhengineering03:05:59

While server is running you can just install

npm i react-modal
then modify the defn app part of main.cljs
(defn app
  []
  (let [modal-state (r/atom false)]
    (fn []
      [:div {:style {:margin "auto"
                     :margin-top "100px"
                     :width "600px"}}
       [:h1 "hi"]
       [:button {:on-click #(swap! modal-state not)}
        "Button"]
       [:> Modal {:isOpen @modal-state
                  :onRequestClose #(reset! modal-state false)
                  :contentLabel "Example Modal"
                  :shouldCloseOnOverlayClick true
                  :style {:content {:top         "50%"
                                    :left        "50%"
                                    :right       "auto"
                                    :bottom      "auto"
                                    :marginRight "-50%"
                                    :transform   "translate(-50%, -50%)"}}}
        [:div
         "This is a modal"
         [:ul
          [:li "With content"]
          [:li "And lists"]]]]])))

dpsutton03:05:08

Again use the slides to see what is new and then check github for the whole file

dpsutton03:05:20

That looks correct

factorhengineering03:05:28

You have cleared that part up for me, thanks. Just trying to help you tweak your guide, if you want, to reach noob-whisperer level. ;)

dpsutton03:05:00

It’s a balancing act. If I put too much on one slide it gets cut off. And spliting over two slides can be confusing as seen early on

jayzawrotny03:05:47

The other nuance is how many people are coming in with no prior dev experience, some prior dev experience, or a lot of dev experience. Currently, I don't know.

factorhengineering03:05:26

Yep. I'm just suggesting that for Slide 16, the text "and change the defn app part of the main.cljs file" might be more valuable than repeating the

(ns app.main
(:require [reagent.core :as r]
["react-modal" :as Modal]))
part. Less text overall to boot.

factorhengineering03:05:37

Same question for the 'Hiccup' section, except it doesn't even have a file extension or a tree diagram to go by.

dpsutton03:05:00

that's explaining the concept of hiccup. the next slide shows the actual hiccup to use and where to save it

factorhengineering03:05:07

Ohhhh. So...there's a file with all html (index.html) and also a main.cljs with some hiccup in place of html sprinkled in it?

factorhengineering03:05:16

Hey, I'm actually going to try walking through that guide of yours now, @dpsutton. Thanks for the clarifications in the sidebar. Quick ultra-basic question (for anyone) before I start: Is there any issue with the top folder name (project name?) containing underscores or spaces? I just have noticed a-lot-of-hyphens-everywhere-in-project-names-in-multiple-online-guides.

dpsutton03:05:43

The top folder is independent of the contents. Spaces in directory and file names seem almost always wrong to me

factorhengineering03:05:18

I mostly live in Windows at the moment, but having used Linux a bunch of times I typically use underscores all over the place instead of spaces in file and folder names (in Windows). I just remember Linux having trouble with spaces.

factorhengineering03:05:57

So...not actually asking if anything would choke on spaces, but actually that question about underscores.

dpsutton03:05:46

The top level directory is independent of the project. It can be named whatever you want

dpsutton03:05:16

Source directories and files need to use underscores due to requirements/limitations of the jvm

factorhengineering03:05:07

underscores or hyphens, right?

dpsutton03:05:12

Underscores. Use hyphens in the namespace names but the files need underscores

factorhengineering04:05:01

😕 Of course it's different. OK thanks.

factorhengineering04:05:18

I am extremely unclear about the term 'namespace' by the way.

factorhengineering04:05:31

I've come across it so much during all the CLJS videos I've watched, but never with any explanation. Is it just variable scope, like within-procedure vs global?

dpsutton04:05:07

read the reference first

factorhengineering04:05:22

That ref is hard to understand. I guess I'll wait until the topic comes up more organically during something I'm doing.

factorhengineering04:05:40

For now my guess is just that namespaces prevent two variables of the same name (maybe one in a library and one in a program using it) from being accessed at the wrong times instead of the other one.

andy.fingerhut04:05:12

@factorhengineering Every Clojure function, etc. is created within a namespace. The built-in Clojure functions are mostly defined in the namespace clojure.core. Some additional string-related ones are in a namespace called clojure.string. Typically when you start a library or application, you create one or more namespaces of your choice, (not clojure.core or anything else used by an existing library you want to use, but it is easy to avoid conflicts), and create your code in them.

andy.fingerhut04:05:47

And yes, the purpose of namespaces is to let you use whatever names you want within your own namespace, and not conflict with the same names chosen in the namespaces created by others.

andy.fingerhut04:05:05

Not everyone does this, but one way to ensure uniqueness of namespacee names is to take a DNS name like http://mydomain.com that you or the organization you work for owns, and use namespace names that begin with com.mydomain, e.g. com.mydomain.mynewlibrary

factorhengineering04:05:15

@dpsutton One more dumb question about your guide: When you say this file is in this folder:

src
└── app
    └── main.cljs
I know you mean \app\src\app\main.cljs because you named your top level folder (and thus your project?) 'app', but...

dpsutton04:05:54

you don't know that

factorhengineering04:05:01

is the 2nd 'app' in the path meant to always be exactly 'app' in any project, or is it meant to be a repeat of the project name (the top level folder name)?

dpsutton04:05:12

it could be FHE/src/app/main.cljs

factorhengineering04:05:47

I mean if I were following your guide exactly, i.e. with mkdir app

dpsutton04:05:47

but this is an important concept called the classpath. by default src is the classpath root. clojure will look for the namespace app.main in a folder called app and a file called main.cljs from the classpath roots

factorhengineering04:05:55

In your case it's ...\app\src\app\main.cljs

dpsutton04:05:07

so for a file to have the namespace app.main, it needs to be at src/app/main.cljs

dpsutton04:05:22

so the first app is irrelevant. if that makes sense

factorhengineering04:05:43

but if the top level folder were foo, would it be \foo\src\app\main.cljs or \foo\src\foo\main.cljs ?

seancorfield04:05:28

That second one would have a namespace of foo.main

factorhengineering04:05:31

Ah. I think you answered it above. it's the 1st one

dpsutton04:05:56

yeah. the namespace and the hierarchy go hand in hand

factorhengineering04:05:00

"clojure will look for the namespace app.main" Got it.

dpsutton04:05:13

and namespaces need hypens but filenames get underscores is the one place they diverge

dpsutton04:05:19

almost always trips people up at least once

factorhengineering04:05:14

and just to be clear, I think I heard that namespaces CANNOT have underscores. It is also true that filenames CANNOT have hyphens, or can those have both?

seancorfield04:05:00

I have to say, the namespace reference is terrible if you're trying to learn Clojure -- it talks about vars and mappings but it says nothing about project structure and how that relates to namespaces.

andy.fingerhut04:05:06

Namespaces can have underscores, as can file names. hyphens in Clojure namespace names are searched for in files that have hyphens replaced with underscores.

dpsutton04:05:07

thanks @andy.fingerhut. I just tested it and namespaces can indeed have underscores but i've never seen anyone do this

andy.fingerhut04:05:31

It is unusual, certainly, but not illegal

seancorfield04:05:35

(I've seen beginners do it "by accident" because they get confused about the rules)

factorhengineering04:05:35

Just to cover file names and folder names, is this correct: • file names can have underscores but not hyphens • folder names can have underscores but not hyphens • namespaces can have hyphens but not underscores (and really the hyphens and just replacements for underscores in the corresponding file names and folder names)

andy.fingerhut04:05:05

namespaces can have either

andy.fingerhut04:05:18

file names that have hyphens will not be found by Clojure when require'ing namespaces.

factorhengineering04:05:36

But don't namespaces essentially refer back to folder names and file names?

andy.fingerhut04:05:46

The file system will let you create such file names, of course, it just isn't productive for Clojure development 🙂

factorhengineering04:05:14

well yes. lol. just talking clojure here

andy.fingerhut04:05:29

This is maybe unnecessary info for you at this point, but you can create namespaces dynamically in Clojure that have no files on the file system to store them.

seancorfield04:05:45

Namespace foo-bar.quux => filename <classpath>/foo_bar/quux.clj (or .cljc or .cljs

andy.fingerhut04:05:52

Don't worry about that too much right now, if it is overload of detail

seancorfield04:05:49

(where <classpath> is going to be your src folder or your test folder typically -- for dev/test they're typically both considered, for production just the src folder, depending on how you set up your project)

andy.fingerhut04:05:52

^ What Sean said

factorhengineering04:05:05

@seancorfieldThat's what I was thinking. Good to see it in an example.

factorhengineering04:05:40

By the way, are the top level folder name and the project name synonymous. I'm guessing yes, but still...

dpsutton04:05:54

i've never made a dynamic namespace. nor worked in a code base where it happened

dpsutton04:05:04

not necessarily

dpsutton04:05:18

notably, you can clone any repo into whatever foldername you want

dpsutton04:05:04

for instance, git clone [email protected]:clojure/core.match.git custom-name will clone the core.match library as custom-name. The project will behave just as if it were saved into a directory named core.match

factorhengineering04:05:58

does top-level-folder-name = project-name ?

seancorfield04:05:11

No, not necessarily.

dpsutton04:05:14

no. as demonstrated above

factorhengineering04:05:21

unless you take specific steps to break that?

seancorfield04:05:26

The "project name" comes into play if you are publishing a library to Clojars/Maven (a repository). Those have a group ID (like your company name or your GitHub username) and an artifact ID (the actual project name). e.g., fhe/my-project, seancorfield/clj-new. That doesn't have to correspond to anything in the source code but it will be part of the descriptor that is used to deploy the project to a repository. It's somewhat common to have the namespace structure of the project match the group/artifact tho'. So the fhe/my-project project would be in a top-level folder called my-project in which would be src and test folders (typically) and inside src you'd have fhe/my_project.clj with a namespace fhe.my-project -- does that help?

seancorfield04:05:06

If you have the :new alias in your ~/.clojure/deps.edn file pointing to the clj-new library, you can see this in play by doing:

$ clojure -A:new app fhe/my-project
That will create my-project/src/fhe/my_project.clj containing (ns fhe.my-project)

factorhengineering04:05:25

Ehh...maybe? Actually I don't get this part: inside src you'd have fhe/my_project.clj

factorhengineering04:05:51

Do you mean inside src you'd have fhe/my_project/src/my_project.clj

dpsutton04:05:56

[email protected] /tmp % clojure -A:new app fhe/my-project
Generating a project called my-project based on the 'app' template.
[email protected] /tmp % tree my-project
my-project
├── CHANGELOG.md
├── LICENSE
├── README.md
├── deps.edn
├── doc
│   └── intro.md
├── pom.xml
├── resources
├── src
│   └── fhe
│       └── my_project.clj
└── test
    └── fhe
        └── my_project_test.clj

6 directories, 8 files
[email protected] /tmp %

seancorfield04:05:08

no. I mean exactly what I say. Try the command above.

factorhengineering04:05:16

Ugh. This thing will not let me edit for some reason.

factorhengineering04:05:00

Trying to edit my post is flash-scrolling me hours back in the chat for some reason.

seancorfield04:05:07

See this:

[email protected]:~/clojure$ mkdir temp
[email protected]:~/clojure$ cd temp
[email protected]U2UJ1N:~/clojure/temp$ clojure -A:new app fhe/my-project
Generating a project called my-project based on the 'app' template.
[email protected]:~/clojure/temp$ tree my-project/
my-project/
├── CHANGELOG.md
├── LICENSE
├── README.md
├── deps.edn
├── doc
│   └── intro.md
├── pom.xml
├── resources
├── src
│   └── fhe
│       └── my_project.clj
└── test
    └── fhe
        └── my_project_test.clj

6 directories, 8 files

seancorfield04:05:29

And

[email protected]:~/clojure/temp$ cat my-project/src/fhe/my_project.clj
(ns fhe.my-project
  (:gen-class))

(defn -main
  "I don't do a whole lot ... yet."
  [& args]
  (println "Hello, World!"))

factorhengineering04:05:17

Actually I don't get this part: inside src you'd have fhe/my_project.clj Do you mean inside src you'd have my_project.clj ?

factorhengineering04:05:54

Is see the big tree you posted. So why is it repeating the top folder name in the subfolder of src??

seancorfield04:05:35

It's a common convention, that's all.

seancorfield04:05:02

Go back and read my explanation, then read the commands I showed and their output. Then go try that yourself.

factorhengineering04:05:09

Also, didn't we just talk above about the fact that the top folder name doesn't matter and is not repeated and clojure looks for app.main (meaning /app/main.cljs)?

factorhengineering04:05:23

I'm getting more confused now,.

seancorfield04:05:31

Right, it is not related. It's just common practice for them to match.

seancorfield04:05:50

Again, read what I said about project names vs namespace structure above.

seancorfield04:05:08

The top-level folder name, the project "name" (group/artifact) and the namespace structure are all independent -- but it is common practice to connect them like this.

factorhengineering04:05:21

What I gather from that was that you could decouple top folder name from project name if you really wanted to.

factorhengineering04:05:27

clojure -A:new app fhe/my-project

seancorfield04:05:32

Only the namespace structure and the file path structure (in src and test) are tied together.

factorhengineering04:05:38

I don't get the line above btw.

factorhengineering04:05:52

Why would you name your app a name with a slash in the middle??

seancorfield04:05:12

Go read the clj-new project README where that is explained.

seancorfield04:05:38

Libraries -- on Maven/Clojars -- have a group ID and an artifact ID and they are separated by a /

seancorfield04:05:35

For example, org.clojure/clojure -- group ID org.clojure (a reverse domain name for ) and artifact ID clojure ... and org.clojure has several projects: clojure, clojure.string, java.jdbc etc.

andy.fingerhut04:05:55

@factorhengineering I have never seen a project or library name with a slash in the middle. Is there something specific that someone else said that leads you to believe they were recommending that?

seancorfield04:05:18

@andy.fingerhut he's misreading the example I gave.

factorhengineering04:05:19

Yes, in seancorfield's example, he has:

factorhengineering04:05:30

clojure -A:new app fhe/my-project

seancorfield04:05:58

So that would be group ID fhe and artifact ID my-project (by convention)

factorhengineering04:05:02

Why not just

clojure -A:new app my-project

seancorfield04:05:36

That is explained the in README for clj-new. Please go and read it.

factorhengineering04:05:31

OK, but if one is not trying to produce a library, and is just working on smaller thing, that slash and 'group ID' would not be needed I guess.

seancorfield04:05:02

It's a good practice to get into.

factorhengineering04:05:42

I have not even heard of clj-new before. I think it's down the reading list for me. 😕 Right now I'm just trying to correctly follow @dpsutton’s guide.

seancorfield04:05:51

If your project name, top-level folder, and namespaces follow that convention, your code is less likely to conflict with code from another library.

factorhengineering04:05:19

THAT statement made complete sense to me.

seancorfield04:05:25

I thought you had talked about clj-new the other day -- when you were asking about deps.edn and creating new projects using that...

factorhengineering04:05:40

If lots of people are uploading libraries there could be conflicts unless the namespaces are unique, so the convention is to have a unique identifier PART of the name (assigned to people or organizations) and a more descriptive PART of the name. That would make sense to me.

seancorfield04:05:05

Libraries have group IDs and artifact IDs -- those must be unique.

factorhengineering04:05:26

so the 2nd part has to be unique too? That makes less sense to me

seancorfield04:05:44

The combination must be unique.

factorhengineering04:05:50

If those are already unique why even have group IDs

factorhengineering04:05:10

OK, so the artifact IDs are NOT, on their own, unique. all good. ok

seancorfield04:05:15

The group ID + artifact ID together must be unique. They are the "coordinates" of the library.

seancorfield04:05:10

group-id/artifact-id -- in Leiningen's project.clj you see [group-id/artifact-id "1.2.3"] for dependencies... and in Clojure CLI / deps.edn it is written group-id/artifact-id {:mvn/version "1.2.3"}

factorhengineering04:05:44

As for whether you mentioned clj-new to me way back, if you did I didn't get that message. All I remember about deps..whatever it's called is that it was a built-in thing that does some of what Leiningen or shadow.cljs does.

andy.fingerhut04:05:06

Sean can correct me if I have this wrong, but I'm pretty sure that the group ID + artifact ID naming scheme was originated by Maven, a tool/system for publishing and distributing Java libraries, that Clojure/Java does not try to replace, but to build on, because it is common to want to use existing published Java libraries in Clojure/Java programs.

seancorfield04:05:11

Inside a library you can have any namespaces -- but again, you want your code to not conflict with someone else's code so following group-id.artifact-id in general is a useful convention (for a file in src/group_id/artitact_id.clj)

seancorfield04:05:06

Right, the two conventions are separate but essentially overlaid in order to keep code from conflicting, both at the library artifact level (group/artifact) and in the Clojure source (`group.artifact`).

factorhengineering05:05:35

OK. Makes sense. Thank you.

seancorfield05:05:37

It would be... somewhat surprising... if you depended on a library as [foo/bar "1.2.3"] and then end up having (:require [quux.wibble]) to bring in its code... library names and namespaces are not usually that disconnected.

factorhengineering05:05:38

Now I feel like I need to pull myself out of this rabbit hole. lol. I mean I'm sure I'll use the info, just not right this second. i want to get through https://github.com/dpsutton/asg-ignite-presentation/blob/master/shadow-cljs.org and maybe https://shadow-cljs.github.io/docs/UsersGuide.html now and then finally actually make something.

factorhengineering05:05:24

I'm crying to actually write some of my own code at this point. lol. So...much...set-up.

seancorfield05:05:33

I think it's important to understand the project basics (for Clojure) in order not to get lost in ClojureScript's ecosystem...

seancorfield05:05:59

It's why I always recommend beginners learn basic Clojure stuff first and then move on to ClojureScript

seancorfield05:05:22

(It's why I also recommend not trying to learn both Clojure and Emacs at the same time)

seancorfield05:05:37

Otherwise you can get rather overwhelmed... as I think you have at times?

factorhengineering05:05:26

So many things to install, so many things to figure out about how to install things, and even with the tools installed this setting up a project or whatever is needed to actually get a cljs file for me to edit in front of me is even more confusing.

seancorfield05:05:56

Yes, there are a lot of moving parts.

seancorfield05:05:17

Clojure isn't "easy".

factorhengineering05:05:53

I understand your Emacs comment, but that bit of learning I willingly signed up for, having tried Emacs for a bit way back. At least it doesn't feel like pandora's box like this set-up stuff does.

factorhengineering05:05:17

Hmm...I guess I mean a Matroska doll

factorhengineering05:05:36

Actually, what I really mean is a hydra. lol

seancorfield05:05:44

Emacs is all of those and more 🙂

factorhengineering05:05:50

I'm at peace with the file opening and saving and weird versions of copying and pasting in it now. I can just use it for that (and CIDER when I figure out what that exactly is) for the moment.

seancorfield05:05:56

I first started using Emacs back in its 17.x and 18.x days... then moved on to other editors and when I learned Clojure (ten years ago) I was using TextMate! After a while I came back to Emacs (23.x I think) for a few years, then switched to Atom in 2015 and that's what I use now.

factorhengineering05:05:12

...and put off learning vim for the moment as well!

seancorfield05:05:39

My knowledge of vim is pretty minimal. I know barely enough for basic editing.

factorhengineering05:05:08

All that investment in Emacs AND you do Clojure and STILL you moved away from it? That's sad to hear, from the perspective of someone who tried hard to identify the best programming language and editor to learn and picked Emacs as the editor. lol

seancorfield05:05:46

About 80-90% of Clojure programmers used Emacs a decade ago when I first picked up Clojure -- but that percentage has dropped slowly each year since I think... I'd have to check the annual survey results...

dpsutton05:05:26

It’s 45/45/10 or so emacs cursive and now vs code

seancorfield05:05:30

Emacs was down to 45% in 2020.

dpsutton05:05:14

But those are of course market share and hopefully the entire clojure user base got bigger :)

seancorfield05:05:50

The 2010 survey suggested about 70% Emacs usage so I was slightly off above.

factorhengineering05:05:04

Oh well. I have tried org-mode and like it a lot. Good tie-breaker.

seancorfield05:05:00

I never got into org-mode but I know people who absolutely love it.

hindol.adhya05:05:52

Org mode can do things few other editors can. Org mode grew organically over time. A new markup/editor will need significant investment before it can reach feature parity.

seancorfield05:05:09

It really just isn't a use case I seem to need. And when I've tried to use it in the past, I just get frustrated with how it works.

hindol.adhya05:05:15

Yeah, that is fare. For me, there are many many useful features, like org capture which let's you capture a thought from any buffer. Deft let's you search all your notes. The table editor is just auto magical. And I also sometimes build presentations in org mode (by exporting to reveal.js).

hindol.adhya05:05:21

Since I am already comfortable with Emacs, I don't need to learn "n" different tools. It is a huge time saver.

seancorfield05:05:40

Yup, totally get that. Emacs is really an operating system that just happens to let you edit files 🙂 I never really got comfortable with Emacs, even using it over decades. I never found a setup I really enjoyed.

hindol.adhya05:05:11

I can suggest Doom Emacs. I find both Spacemacs and Doom enjoyable. They both come with sane configurations. These are recent projects and maybe you have not tried these.

seancorfield05:05:21

I do see value in an editor that works in a bare terminal but I rarely need that low-level stuff these days -- I can connect a REPL to any server, any process; I can connect Atom to that too. I can do everything from Clojure, in my editor, But when I'm not "programming" I like modern UI/UX tools -- so much so that my non-programming activities are all done in Windows or MS apps on my iPhone, my Windows laptop, and even my Mac desktop (which runs Windows 10)...

seancorfield05:05:02

Not interested. I have a lot of REBL integration invested in Chlorine now and use bare socket REPL everywhere. I don't want anything to do with CIDER/nREPL.

hindol.adhya05:05:07

Yeah, once you are already invested in something, there needs to be a solid reason to switch.

hindol.adhya05:05:16

I totally get that.

seancorfield05:05:19

And I love REBL 🙂

jayzawrotny05:05:51

I get the love but thankfully my favorite Notebook app, Notion, just became free for personal use.

orlandomr2705:05:44

Emacs dropped slightly to 43%, IntelliJ/Cursive rose slightly to 32%, and VS Code with Calva had the biggest increase to 10%.

seancorfield05:05:15

Yeah, Atom is still very much a minority choice but I really do like it. And I love that the latest version of Chlorine (for Atom) lets me customize it via ClojureScript (using sci so there's no compile step and changes apply immediately).

jayzawrotny05:05:02

I used Atom + proto repl for a year or so before Spacemacs clicked and became my favorite. It was pretty good overall and Chlorine seems even better from what I’ve seen.

seancorfield05:05:16

Yup, I initially used ProtoREPL and loved it. I was disappointed when Jason moved on. I was VERY excited when Chlorine appeared and I love the inline eval results (just like ProtoREPL) and the socket REPL integration -- so I can connect Atom/Chlorine to any process locally or remotely (since socket REPL is built-in to Clojure and just JVM opts are needed to start a socket REPL on any process).

seancorfield05:05:55

Chlorine recently introduced interactive rendering (basically Hiccup and ClojureScript to power React components that render inline in the editor!).

seancorfield05:05:26

And then the full customization via ClojureScript -- so all my REBL integration is done that way now.

factorhengineering05:05:43

@jayzawrotny I was looking hard at SpaceMacs until at the 11th hour Icame across a bunch of comments that Doom was similar but much faster.

factorhengineering05:05:20

Do you use EVil? Itried vim for a couple of days a year ago. Even more of a mindmelt than Emacs but maybe worth it in the end.

factorhengineering05:05:52

...like Dvorak typing which I'm very glad I switched to years and years ago.

jayzawrotny05:05:23

Yes, I use Evil mode. I learned vim 10 years ago and always enjoyed editing that way. That's true DOOM is probably more performant but I think Spacemacs is more batteries included and a bit easier to use. Though from what I understand using DOOM forces you to learn more about how to work with Emacs' internals so that's not a bad thing.

factorhengineering05:05:22

What aspects of spacemacs are more 'batteries-included'? Really interested, since I'm just barely starting with Doom and could possibly switch.

jayzawrotny05:05:06

Comes with fuzzy search, modern UI features, integrates auto-complete and helm (searchable menu system) into everything and there’s a strong consistency with how everything is accessed. The biggest add are the layers, which are groups of emacs packages that solve bigger problems. For instance there’s a Clojure layer that includes cider, clojure syntax, clojure key bindings, and clojure formatting https://www.spacemacs.org/layers/+lang/clojure/README.html

factorhengineering06:05:27

I'm not very familiar with the other things on your list, but I do know Doom has the ones I recognize: helm (or maybe ivy which is like helm?) and...i'm not sure if they're called layers in doom, but I think it turned on a bunch of clojure things when I went to clojure...mode? (mode might not be the right word)

jayzawrotny06:05:34

That’s good to know. Clojure-mode is the right word but it’s only one package I think, the spacemacs layer includes clojure mode among a few others.

jayzawrotny07:05:46

Ah Doom has modules instead of layers. It’s less polished in the UI\UX. It’s less streamlined so it may take more config to make it the way you like. Spacemacs is more about reasonable defaults and is more like a complete framework that don’t want to get stuck customizing Emacs too much. https://youtu.be/6xKzrcrv_fU

jayzawrotny07:05:09

But Doom is more about customizing the experience to your liking

hindol.adhya10:05:19

While Doom makes customization much easier as compared to Spacemacs, it works out of the box too.

hindol.adhya10:05:29

All the benefits you mention about Spacemacs is there in Doom too. The Clojure module includes CIDER, clj-refactor, key bindings.

factorhengineering05:05:51

Is there any way to export chat logs from this? I'm trying to get back and look at the answers about filename underscores (yes I'm already fuzzy on the details!), but this interface keeps unloading older chat messages. I hate that. It's all or mostly just text! Keep it all in memory!

seancorfield05:05:23

All the chat here is mirrored to Zulip and also to the ClojureVerse log... I'll get you the URLs...

factorhengineering05:05:17

So...Zulip is easier (possible) to export from compared to Slack?

factorhengineering05:05:50

Is it better? Are you on that too? I don't really want to sign up for yet another thing. lol

seancorfield05:05:21

Yes, I'm on Zulip. It's much lower traffic than Slack. But it has unlimited archive/search.

seancorfield05:05:41

Many channels here are archived to Zulip (it's one-way: what you post on Zulip does not appear here)

seancorfield05:05:24

Some channels are mirrored to Matrix/Riot as well but I think that's more focused on mirroring IRC which still has hundreds of active users.

seancorfield05:05:43

(and there's a Discord community, and Telegram and various others 🙂 )

seancorfield05:05:28

FWIW, I login to Zulip using GitHub so it's not like I had to signup for anything new really.

factorhengineering05:05:05

Thanks for the log link. Much more searchable (though lists and probably other things are messed up by the logging.).

factorhengineering05:05:20

also the IRC tip. I have HexChat open already just to get general Emacs help. Didn't know I more clojure goodness was just a /join away.

factorhengineering05:05:12

Note sure if @dpsutton is still here, but I just ran this and got this, which doesn't lookequite right:

factorhengineering05:05:33

npm i react create-react-class react-dom shadow-cljs npm WARN deprecated [email protected]: [email protected]<3 is no longer maintained and not recommended for usage due to the number of issues. Please, upgrade your dependencies to the actual version of [email protected] npm notice created a lockfile as package-lock.json. You should commit this file. npm WARN [email protected] No description npm WARN [email protected] No repository field. + [email protected] + [email protected] + [email protected] + [email protected] added 119 packages from 191 contributors and audited 119 packages in 10.867s 1 package is looking for funding run npm fund for details found 0 vulnerabilities

dpsutton06:05:18

There’s a warning but I’ve been ignoring it. The other warnings are warnings about your own package.json file. About no description and repository information. Completely benign

hrmeyer11:05:13

I have a beginner concurrency question. I'd like to update a Java array faster. Right now I do this:

(doseq [i (range (count pxs))]
        (aset-int pxs i (nth pxs-sorted i))
        (when (= (rem i 50000) 0)
          (println i)))
      (println "done setting pixels")
But it takes several minutes (the array has length 360.000). My idea was to have several threads do it at the same time, but as I'm new to Clojure concurrency, I'm not really sure where to get started. Ideally, I would like to have the content of the doseq be done with many threads, but resuming the program after the loop is done being processed.

codestar812:05:47

Does pxs-sorted rely on accessing the entire array of pixels? Or can you break up the array into separate sub arrays so you could process it in parallel?

potetm12:05:46

I dunno what pxs-sorted is, but nth is probably doing a linear scan on it.

potetm12:05:36

other micro optimizations: use dotimes to avoid allocating a seq, use zero? instead of (= … 0)

potetm12:05:14

re: parallelization: that’s risky with a backing array, but pmap is an easy thing to try.

codestar812:05:18

Ya. Probably linear, and needs to load entire array to sort it.

codestar812:05:52

Yep. I was thinking pmap to parallelize. But probably can't because of the sort algorithm.

codestar812:05:12

Divide and conquer, parallelize

codestar812:05:29

O(Log n) complexity instead of linear O(n)

hrmeyer12:05:43

Oh, that "nth" is linear, I didn't think of! I guess get is better for arrays? Actually, I tried pmap now. It helps quite a lot. The computation took ~2 minutes instead of 7 minutes, with correct result. (though I still think 2 minutes is quite long for just putting values inside an array...)

codestar812:05:22

I'm super newb at concurrency. It's interesting... What other ways could you try? Agents or futures? https://purelyfunctional.tv/guide/clojure-concurrency/

hrmeyer12:05:10

Hmm, I get nil when trying to use get instead of nth ... I think its because seqs dont have indexes maybe?

hrmeyer12:05:10

Ah!!! That was it. Doing a combination of pmap and to-array in the beginning got the time down to <1 sec

codestar812:05:19

Can you post pxs-sorted body?

codestar812:05:51

Because I'm actually surprised pmap worked on a sort

hrmeyer12:05:29

The reason it works is that pxs-sorted is not changed. All I want to do is to copy the contents of pxs-sorted into pxs in the same order, in place, which was kinda clunky in clojure...

(let [pxs (q/pixels)
          pxs-sorted (to-array (sort-by q/brightness pxs))]
      (time
       (doall (pmap (fn [i]
                      (aset-int pxs i (get pxs-sorted i))
                      ) (range (count pxs)))))
      (println "done setting pixels")

hrmeyer12:05:04

Oh! Just noticed the pmap is slower! So actually, my biggest mistake was using nth...

codestar812:05:38

Ahh. So it's not actually using a sorting algo. Gotcha.

codestar812:05:03

Clojure is such an elegant lang.

potetm16:05:57

if it’s an array, aget is better

potetm16:05:09

but i dunno why you need to array that – it’s immutable anyways

potetm16:05:30

arrays help for various reasons, but chief among them is fast mutability

hrmeyer06:05:46

@ I think I had to do it because sort-by returns a seq and they don't have indexes. Unless I have misunderstood something, of course.

potetm12:05:52

@ Vectors have constant-time, indexed access. If instead you do (vec (sort-by q/brightness pxs)) you’ll get roughly the same lookup performance.

neo255116:05:06

I would be interested in training my skills with clojure.test.check, and I have trouble to train my skills at writing properties. I read Clojure Applied from Alex Miller, watch the talks from the conj and from John Hughes at Clojure West and the latest one about pure function (how to specify it). Does anyone know if there is a repository (or something else) where there would be some faulty code and we would use test.check to write properties to find all the possible bugs?

andy.fingerhut18:05:59

First, I would say that if you are thinking of property based testing as "a way that if done properly, can guarantee finding all of the possible bugs", then I would mentally take a step back from that. Why? Because it is still a form of testing, albeit one where a myriad of test cases you might never think to write will be automatically generated for you.

andy.fingerhut18:05:12

Testing can find the presence of bugs. It will not prove the absence of bugs.

andy.fingerhut18:05:53

I think it is an amazingly good testing technique to use, and can greatly increase your confidence in how well tested the code is.

andy.fingerhut18:05:39

But as a very simple example, suppose you were testing a function to sort a vector of numbers in increasing order, and your postcondition checking code verified that the returned value was a vector that was the same length as the input vector, and did no other checks. You could use that as a test on a function that simply returned the input vector, and the automatic generation could generate thousands of test cases for you, and it will never find the obvious bug that the returned vectors are not sorted.

andy.fingerhut18:05:05

Determining what properties to check is part of the art of property-based testing. How complete/thorough do you want to be there?

neo255119:05:32

This is exactly the kind of experience and sensibilities I want to acquire. I totally agree that it is not a way to find all possible bugs, but more a way for verifying assumptions I have about my code (and also checking when I update my deps).

neo255119:05:05

And yes, knowing which properties to test is also what I would like to learn 🙂

neo255119:05:01

I had in mind, there should be some projects/repository with intentional bugs and the goal would be to find them thanks to either PBT.

neo255119:05:52

(As for for sorted collection, it is a textbook example of recursive properties, but I would like to learn the tool more than having a unrationnal desire to apply it).

andy.fingerhut20:05:27

If there is such a learning resource, I have not heard of it, but yes that would be nice

neo255120:05:13

I think it would be a good resource honestly. Learning to debug probably belongs to the most underrated skills.

alidcastano17:05:27

i’m writing a macro that I want to accept a quoted list, e.g. '[Foo] , but because the macro is receiving the reader form (quote [Foo]) i’m iterating over it incorrectly. What would be right way to grab the list? I tried unquote but it gives me unbound function error, so doing this and it works (if (= (first tags) 'quote) (second tags) tags)

seancorfield17:05:27

@alidcastano That is correct. ' is just a reader shorthand for (quote ,,,) so the form is actually read in as (quote [Foo]) -- macros do not evaluate their arguments.

seancorfield17:05:47

You could just pass [Foo] and your macro would not evaluate it anyway.

alidcastano17:05:32

the linter gives me an unresolved symbol error, which is why I’m quoting it. so I’ll just stick with grabbing the desired form (if (= (first tags) 'quote) (second tags) tags)

seancorfield18:05:47

You could pass '[Foo] to a function and it would do what you want.

seancorfield18:05:49

Or you could tell the linter (`clj-kondo`?) about your macro so it doesn't flag the unresolved symbol. Ask in #clj-kondo for more information about that.

dpsutton18:05:11

This is certainly an area where you know more than the longer. You are creating syntax so the linter is irrelevant here.

seancorfield17:05:36

(the first rule of Macro Club is Don't Write Macros -- so be sure that what you're trying to do can't already be written as a function instead of a macro)

yokoyama.km20:05:17

Hi, looking for an advice: should I use clojure.spec or stick with plumatic/schema on new projects?

seancorfield20:05:17

@yokoyama.km Personally, I'd recommend clojure.spec -- it is useful for a lot more things, in my opinion, and it's also "official" since it's from the core team.

neo255120:05:42

Is spec2 compatible with specs.alpha?

seancorfield20:05:48

We use it heavily at work and I wrote up the various different ways we use it https://corfield.org/blog/2019/09/13/using-spec/

smith.adriane20:05:28

I noticed that you cite “Deriving code from specs” in your post. how is the inspectability of spec? do you rely on stuff like (= (:pred s) int?) ?

smith.adriane20:05:25

I assumed that since spec is predicative and functions in clojure are fairly opaque, that the specs themselves would be fairly opaque

seancorfield21:05:50

You can get the form of a Spec and then you can walk the form, yes.

seancorfield21:05:12

For example, we have s/keys specs for database records and from the form of those specs we derive a series of CRUD functions automatically.

seancorfield21:05:40

We use a similar technique to turn explain-data into user-facing error messages, based on partial descriptions of predicates that may fail in the Spec. For example, knowing that contains? is used to detect required keys in hash maps, lets you identify missing parameters/keys in maps automatically from the spec failure.

smith.adriane21:05:04

I guess I was just curious how you handle using a spec like

(s/def ::error int?)

smith.adriane21:05:36

if int? is a function, all you can do is call it (not super useful if you don’t know what it is) or test for equality

smith.adriane21:05:02

so I guess you have to rely specs using predicates that you have access to

smith.adriane21:05:46

so in this case, (s/def ::error #(int? %)) would be far less useful from the perspective of using specs to generate code?

seancorfield21:05:22

It's still just code-as-data -- you can walk into it to get at the predicate

Clojure 1.10.1
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/def ::error1 int?)
:user/error1
user=> (s/def ::error2 #(int? %))
:user/error2
user=> (s/form ::error1)
clojure.core/int?
user=> (s/form ::error2)
(clojure.core/fn [%] (clojure.core/int? %))
user=>

seancorfield21:05:24

We look for clojure.core/fn in the first slot of a sequential form and then take the last form out of it and inspect that.

smith.adriane21:05:12

👍 good to know

seancorfield21:05:16

"It's just data" 🙂

seancorfield21:05:47

We actually have a function that walks through functions and expressions to find the most likely predicate and its arguments, e.g., #(contains? % :foo) would return [clojure.core/contains? :foo] as a tuple. It's "accurate enough" for the heuristics we need to turn spec failures into predicate tuples that we then use as keys in maps to messages (we try the full tuple first, then drop arguments from the tuples, finally trying just the predicate itself to find a match).

seancorfield20:05:15

Spec 2 isn't directly compatible with clojure.spec.alpha (aka Spec 1) but Spec 2 isn't ready for use yet

srijayanth20:05:33

@seancorfield - thanks. This is a useful article

prnsml20:05:35

(ns me.specs-playground
  (:gen-class)
  (:require [clojure.spec.alpha :as s]))

(s/fdef print-digit
  :args (s/cat :just-an-int int?))
(defn print-digit
  [just-an-int]
  (println (str just-an-int)))


(defn -main
  [& _]
  (print-digit "asds"))
I'm playing with specs right now, and here ^ I have simple example where I added specs for print-digit function. Is there a linter which would catch that I'm calling print-digit with a wrong argument (String instead of intiger)? Or specs are only useful for writing tests at the moment?

seancorfield20:05:09

@prnsml argument-checking is enabled by calling instrument

yokoyama.km20:05:34

Thank you, @seancorfield. And should new projects use Spec 1 or Spec 2 (although it is considered not ready for production yet)?

seancorfield20:05:33

Spec 2 is still changing a lot. It's not really ready for any usage yet. Unless you are deliberately exploring/testing it to provide feedback to the core team 🙂

qmstuart20:05:09

Anyone got any experience of CORS for CLJS using cljs.ajax to a http-kit server

qmstuart20:05:24

what libraries I should be looking at, or just anything I can read at all to help

patrick.glind20:05:22

Question, I’m trying out several JSON packages, however only with org.clojure/data.json decoding i can easily translate datetime string to actual DateTime

patrick.glind20:05:41

what is the idiom to do this in Clojure for other packages?

patrick.glind20:05:44

should I first just import the full data set and then do updates in the data set after the fact?

patrick.glind20:05:42

example being cheshire*

seancorfield20:05:54

@patrick.glind Each library will have its own mechanism for controlling date conversions.

patrick.glind20:05:24

the weird thing is, I started a new clojure app to try out all kinds of JSON libraries

patrick.glind20:05:51

however don’t really see the option to convert decoding json data to the right ‘type’

patrick.glind20:05:28

with clojure/data.json I can do this:

(json/read-str json-file
:value-fn json-value-reader ; each key/value will go through this function :key-fn keyword)

patrick.glind20:05:55

but for cheshire and jsonista

patrick.glind20:05:59

i don’t see that kind of feature

didibus20:05:09

JSON doesn't support dates as a datatype

didibus20:05:28

You need to use your own scheme on top

patrick.glind20:05:36

yeah, it comes in as a String

patrick.glind20:05:47

i just want to immediately convert it to DateTime

patrick.glind20:05:55

same as I would do with a CSV file

patrick.glind20:05:14

but then i figured, like with data.json, there should be an idiomatic way

patrick.glind20:05:17

to convert on import

kelveden20:05:19

The problem is that there isn't a objectively "correct" way of knowing what date/time type a JSON string represents. For a start, there are several different ways of representing dates in the JVM.

seancorfield20:05:25

According to the Cheshire readme, the decode function can take a custom decoder that handles custom parsing based on the field name

seancorfield20:05:52

In 2.0.4 and up, Cheshire allows passing in a function to specify what kind of types to return, like so:

;; In this example a function that checks for a certain key
(decode "{\"myarray\":[2,3,3,2],\"myset\":[1,2,2,1]}" true
        (fn [field-name]
          (if (= field-name "myset")
            #{}
            [])))
;; => {:myarray [2 3 3 2], :myset #{1 2}}

didibus20:05:56

I don't know if it changed since that ticket.

patrick.glind20:05:25

@didibus, what is the clojure idiomatic way to convert it after import?

patrick.glind20:05:38

@seancorfield, yeah I saw that in the doc’s

patrick.glind20:05:55

[key-fn array-coerce-fn]

patrick.glind20:05:10

i tested to see if i could make it work, but no luck

seancorfield20:05:32

Oh, I missed that part. Sorry.

patrick.glind20:05:48

i totally understand if the packages don’t convert the data, just do the import part

patrick.glind20:05:59

i’m just curious how to process it afterwards idiomatically

didibus20:05:36

Hum.. are you, actually reading the doc, are you sure it's not for map?

didibus20:05:18

Is this json you get from elsewhere? Or you encode it yourself?

patrick.glind20:05:27

i get it from elsewere

patrick.glind20:05:59

parsing is no problem with all 3 libraries i’m working with

didibus20:05:02

The best would be to encode the date to a map with a custom encoder. And then I think you can use that custom decoder function to decode it back to date

didibus20:05:35

From elsewhere, hum, so how can you tell the difference between a string with a date and just a string of text?

patrick.glind20:05:37

it’s just that so far, only data.json supports converting types on import

patrick.glind20:05:57

the key is :created_at (i convert to keyword)

didibus20:05:18

In any case, you can post process it with clojure.walk

patrick.glind20:05:35

"created_at": "2018-04-24T14:51:11.220-04:00"

didibus20:05:03

Oh I see, then what Sean said from the doc, you sure it doesn't work?

patrick.glind20:05:07

i’m able to convert it with this:

patrick.glind20:05:09

(defn json-value-reader [key value]
  (if (= key :created_at)
    ;; (java.sql.Date/valueOf value)
    ;; (java.time.LocalDate/parse value)
    ;; (read-string (str "#inst" value))
    (java.time.ZonedDateTime/parse value) ; 
    value))

seancorfield20:05:18

The Cheshire readme is confusing -- decode is just an alias for parse-string and that only works for arrays.

seancorfield20:05:37

I would just stick to clojure.data.json then.

patrick.glind20:05:54

yeah, i was getting to that point

patrick.glind20:05:07

but was hoping since i’m a beginner, it would be me that’s missing something haha

patrick.glind20:05:40

my next thought was: after import: just do a mass ‘update’ on the data, convert the string->ZonedDateTime

patrick.glind20:05:53

but wanted to find the idiomatic Clojure way

didibus20:05:56

I think that's your best bet

didibus20:05:17

You can do a clojure.walk post processing pass I guess

patrick.glind20:05:29

i’ve never worked with that

didibus20:05:34

Unless we're missing an obvious feature in Cheshire

patrick.glind20:05:07

it’s the same with jsonista

patrick.glind20:05:43

for decode there are very few options

seancorfield20:05:58

As a beginner, it's reasonable to use the org.clojure libs as a first step if they do what you need. Otherwise you can get overwhelmed by all the choice (Clojure has so many libraries for any given problem).

patrick.glind20:05:39

the weird thing is, i was trying to use the org.clojure library for parsing a ~30mb XML file

patrick.glind20:05:49

however that caused a lot of performance trouble ha

seancorfield20:05:50

Ah, and it was very slow?

kelveden20:05:52

I'm not suggesting using it (definitely stick with clojure.data.json for now) but something I've used before with some success for this sort of thing for more complex data structures is Metosin's spec-driven transformations https://github.com/metosin/spec-tools/blob/master/docs/10_spec_transformations.md

patrick.glind20:05:13

org.clojure for json works fast enought for me 🙂

seancorfield20:05:15

Yeah, well, you're going to trade off performance for functionality in a lot of situations I suspect 🙂

patrick.glind20:05:23

haha true true

patrick.glind20:05:05

btw @seancorfield, i liked your post on next.jdbc, next time i work with a database i will definitly try your library!

didibus20:05:03

I feel we are missing something. Kind of surprised Cheshire and jsonista wouldn't support custom decoder/encoders at this point. Doing a post processing pass seems way slower, and they're all about performance

patrick.glind20:05:29

i was thinking the same thing, confusing..

seancorfield20:05:17

Probably because they're both based on Jackson, a Java library, and that doesn't expose the feature -- or they're both so focused on performance that they don't want arbitrary value transformations?

patrick.glind21:05:42

i will try some more things

patrick.glind21:05:46

thanks for your help

patrick.glind21:05:10

1 thing that is very confusing to me as a beginner though is how the clojure.xml library is having terrible performance for me

patrick.glind21:05:18

like i said, it’s a ~30mb file

patrick.glind21:05:29

and i can read it, run parse on it

cdimara21:05:30

When you recur a function you're not creating a new thread, so a with-local-vars binding will persist through recursive calls correct?

didibus21:05:37

I mean, to be fair, that's pretty big :p

didibus21:05:59

Lots of editors would choke on that size of file

patrick.glind21:05:06

haha, alternatively I found BaseX which parses it in split second and I can run XQuery commands on it

patrick.glind21:05:25

yeah, i understand that parsing/reading the file can take some time

patrick.glind21:05:52

but for some reason, afterwards I can’t really work with the XML as it brings my REPL to a halt

patrick.glind21:05:56

maybe i’m doing something wrong

dpsutton21:05:20

(cheshire/parse-string
 (cheshire/generate-string {:bar (java.util.Date.)}))
this points and the json spec confirms that there's no such thing as date json right?

patrick.glind21:05:53

@dpsutton as far as i know, it’s a string

patrick.glind21:05:25

what is the idiomatic way to ‘walk’/‘process’ an XML file with Clojure?

patrick.glind21:05:37

should i try to think about it like a map like dataset

patrick.glind21:05:46

like the clojure.zip library

patrick.glind21:05:53

where you go through the dataset

patrick.glind21:05:26

where you do like (zip/down), (zip/right) etc

didibus21:05:38

Data.xml is built on top of Java Stax library

didibus21:05:09

Zippers are nice, but need to load the whole thing in memory

didibus21:05:29

So 1mb of XML probably becomes 10mb once loaded in memory for zipper

didibus21:05:53

Data.xml doesn't load it all in memory, by using Stax under the hood

didibus21:05:08

But how you query the XML will matter

patrick.glind21:05:30

i was googling for an XQuery library earlier today to work with in Clojure ha

didibus21:05:32

If you do a big search, even though it's lazy, it'll still need to load everything it searches over

didibus21:05:22

So it might be related to how you're querying over it, for the slowness I mean

didibus21:05:54

Also, at the REPL, trying to print anything too big will make everything super slow, your terminal/editor cannot handle large amount of text

patrick.glind21:05:03

latest thing i am considering is creating a server, for example with BaseX, which takes the xml file as a database, and then just query an API with clojure for the right data

patrick.glind21:05:13

yeah, i noticed that ha!

patrick.glind21:05:32

unfortunately even running (first …) on the xml file after the 1st tag

patrick.glind21:05:35

halts the REPL

didibus21:05:54

How big is that tag?

didibus21:05:28

The thing is, the Repl will keep trying to print things, and printing lazy things realizes them

patrick.glind21:05:10

i don’t know the size

didibus21:05:19

So if (first tag) returns a LazySeq for example, or something lazy, and you print that, it'll recursively realize it all and print everything

didibus21:05:02

Try:

(def ftag (first xml))
(type ftag)

didibus21:05:04

I've never used data.xml, but I think from the doc, you would call parse and then use either tree-seq or postwalk over it to extract what you want

patrick.glind21:05:06

am trying this now

patrick.glind21:05:05

(type (first xml-raw))
clojure.lang.MapEntry

patrick.glind21:05:22

reading the file this way: (def xml-raw (-> filename io/file xml/parse))

patrick.glind21:05:45

i tried the example from http://clojure-doc.org/articles/tutorials/parsing_xml_with_zippers.html, with io/resource, however that returned nil so didn’t work

patrick.glind21:05:03

(count xml-raw)
3

patrick.glind21:05:26

(first xml-raw)
[:tag :datafile]

patrick.glind21:05:35

(second xml-raw)
[:attrs
 {:xsi:noNamespaceSchemaLocation "wiitdb.xsd",
  :xmlns:xsi "",
  :xmlns:xs ""}]

patrick.glind21:05:52

opening the 3rd brings the REPL to a halt ha

patrick.glind21:05:57

so i can’t show that one

didibus21:05:14

What is the operations you're looking to do? Just like explore the document from the Repl?

patrick.glind21:05:57

at first i was trying to do that yes, but quickly found out that wouldn’t work

patrick.glind21:05:13

now i just want to convert the XQuery I created into a Clojure ‘function’ or somthing

didibus21:05:21

I'd say if that's the case, for big XML files, tens to hundreds and above megabytes, probably using Berkeley DB XML would be best.

patrick.glind21:05:31

with learning clojure, i’m taking every opportunity to use it ha

patrick.glind21:05:37

so i found this ~30 mb xml file with Wii games data and figured, let’s create an app ha

didibus21:05:58

For a single query over the data, you should be able to use data.xml. But the challenge is writing an efficient query for it that only pulls in the minimum needed

patrick.glind21:05:08

so here is the working XQuery:

let $search-for := 'mario'

for $game in /datafile/game
where contains(lower-case($game/@name), $search-for)
where $game/type = ''
order by $game/@name

return <a href=''>{data($game/@name)}</a>

didibus21:05:14

And at the REPL, you will constantly be at risk of accidentally printing large amount of the xml

patrick.glind21:05:34

yeah, thankfully in Calva i can interrupt running evaluation, saved me several times ha

didibus21:05:05

I think you should be able to recreate that query with that

patrick.glind21:05:57

that looks very interesting, i will try right away

patrick.glind21:05:10

i get the feeling this library is out of date, https://github.com/clojure/data.xml

patrick.glind21:05:29

i know that Alex was updating all the Clojure libraries to 1.0.0 that were working some time ago

patrick.glind21:05:46

this one is on 0.0.8, it might not be stable yet

seancorfield21:05:01

@patrick.glind use the 0.2.0 alpha version

seancorfield21:05:46

If you're used to versioning in other tech, Clojure's approach can look very strange

patrick.glind21:05:01

i am using deps.edn, can’t seem to be able to find the library on Clojars

patrick.glind21:05:09

ok i will use that

patrick.glind21:05:32

yeah i read about that some time ago, also that in Clojure land, last commit > 1 year ago doesn’t mean out of date haha

seancorfield21:05:54

Contrib libraries are not on Clojars. They're on Maven.

seancorfield21:05:18

org.clojure/data.xml {:mvn/version "0.2.0-alpha6"}
That's in our production code at work.

qmstuart21:05:53

why is their so many options for everything in clojure

seancorfield21:05:54

(we switched from lein to boot in 2015 and then to CLI/`deps.edn` in 2018)

qmstuart21:05:59

it's really not helpful for beginners.

qmstuart21:05:07

I feel a curated list of libraries for common tasks is needed.

patrick.glind21:05:25

yeah i was going to say the same thing haha

patrick.glind21:05:49

my ‘workflow’ is usually to go to clojars to find the actual deps.edn ‘line’ to use

qmstuart21:05:12

I have a bunch fo things I want to build, and I keep trying to build them in Clojure, but invariably hit a problem where i have no idea what library to use or some tooling just fails and somethjing that I could do in an hour or so ends up with me losing hours and hours or days trying to get it working in Clojure and I just give up

qmstuart21:05:10

and I go do it in C#

seancorfield21:05:14

@qmstuart Clojure is not easy. It's simple, but simple is not the same as easy.

seancorfield21:05:27

Clojure is hard to learn and hard to master. It isn't for everyone.

qmstuart21:05:56

I think it's mostly hard to master because the quality of documentation is quite poor though

seancorfield21:05:32

There was a lot less documentation when many of us learned it in the past. It was even harder to learn then.

seancorfield21:05:07

There are so many books, tutorials, and videos nowadays by comparison.

qmstuart21:05:30

Yes, there is a lot of documentation, but a lot of it is quite shallow I find

seancorfield21:05:56

It's OK for something to be hard to learn/master. A violin is hard, for example (a common comparison with Clojure as a "sharp tool").

smith.adriane21:05:28

I haven’t had any issues with documentation quality. I think one of the hurdles is that the documentation might be for java code that a clj library is wrapping (or js code that cljs is wrapping)

smith.adriane21:05:55

so it’s not always clear where to look, but I think I’ve gotten so used to it, that’s not really a pain point for me personally

seancorfield21:05:21

There is certainly an underlying assumption behind a lot of the material that the developers have a fair bit of programming background. But it's slowly improving over time.

smith.adriane21:05:40

being familiar with both java and js can provide a huge boost for using clojure/clojurescript

seancorfield21:05:37

Anyways, in case it was missed, I posted a link to http://clojure-toolbox.com which is a (lightly) curated list of libraries organized by category, in case that's helpful to either of you @patrick.glind or @qmstuart

didibus21:05:47

@patrick.glind This tutorial seems pretty good. And it shows how to parse and query a 44GB XML file in Clojure: https://blog.korny.info/2014/03/08/xml-for-fun-and-profit.html

patrick.glind21:05:21

awesome, thanks!

patrick.glind21:05:24

i will look at that tomorrow

patrick.glind21:05:33

i was able to get part of the way working

patrick.glind21:05:53

although the REPL crashed on xml-seq while following https://clojuredocs.org/clojure.core/xml-seq

patrick.glind21:05:39

(ns wiitdb
  (:require
   [ :as io]
   [clojure.data.xml :as data-xml]))

(def filename "wiitdb.xml")

(def xml-raw (data-xml/parse (io/reader filename))) ; this works..

(xml-seq xml-raw) ; crashes repl

patrick.glind21:05:46

this is what i have now

seancorfield21:05:29

I'd expect that to crash the REPL: you're asking it to print (and therefore realize) the entire lazy sequence.

didibus21:05:32

Yup, you basically can never let the Repl print the XML or any big parts of it or it'll choke

seancorfield22:05:14

You could probably print (first (xml-seq xml-raw)) or maybe even (take 5 (xml-seq xml-raw))

didibus22:05:27

The tutorial I linked has a to-short-xml convenient util that will print the node without it's children which you can use

patrick.glind22:05:45

@seancorfield yeah that works, doing take 5

patrick.glind22:05:57

@didibus, it’s on my todo list for tomorrow’s reading 🙂

patrick.glind22:05:02

thank you guys so much for your help!

patrick.glind22:05:17

it’s bedtime here in Netherlands haha