Fork me on GitHub

2. Beyond installing Node.JS (and NPM), the guide suggests just running the following to get shadow.cljs:

npx create-cljs-project my-project
and I did that. I do now see "For convenience you can run npm install -g shadow-cljs or yarn global add shadow-cljs. This will let you run the shadow-cljs command directly later. There should always be a shadow-cljs version installed in your project, the global install is optional." Is that a good idea?

eccentric J00:05:25

I don't think that's really necessary for now


A bit farther down I see "If globally, you can use the shadow-cljs command directly.

$ shadow-cljs help
" and "The guide will assume there is a global install to keep examples short but this is not required."

FHE00:05:38 really seems like I should do that global installation step.

eccentric J00:05:46

If you don't go global, you can just use npx shadow-cljs help


I had the same confusion in that guide


What are the advantages / disadvantages of npx v global?


I was confused as I'd never used npm or NodeJS before


I usually install it as a dev dependency in any project

eccentric J00:05:30

Well the big disadvantage of global is that if you work in various projects, they may rely on a specific version of shadow-cljs


I do see the suggestion for npx shadow-cljs help. but if I run that I get:....


npm ERR! code E404 npm ERR! 404 Not Found - GET - Not found npm ERR! 404 npm ERR! 404 'shadow.cljs@latest' is not in the npm registry. npm ERR! 404 You should bug the author to publish it (or use the name yourself!) npm ERR! 404 npm ERR! 404 Note that you can also install from a npm ERR! 404 tarball, folder, http url, or git url. npm ERR! A complete log of this run can be found in: npm ERR! C:\Users\Ady\AppData\Roaming\npm-cache\logs\2020-05-23T2358_07_860Z-debug.log Install for [ 'shadow.cljs@latest' ] failed with code 1

eccentric J00:05:39

I think you typed shadow.cljs not shadow-cljs




but still...the entire guide is written assuming the 'global installation', so...sounds like I should.

eccentric J00:05:29

It's up to you, if that would make it easier to follow along it's worth it.


Global is more convenient, and for a tool you want to install, I feel it makes sense. You can re-install things as local later if you need a specific version for a specific project I believe


Makes sense. I will. I had a 3rd what was it?...


Oh yeah...


3. The quick start guide reads that you need Java 8 and teh cljs.jar file


I asked here where I supposed to put the cljs.jar file, and someone told me I actually shouldn't download it.


So I skipped that and by the time I had installed Node.JS (and NPM) and shadow.cljs (by creating a project, so not actually installed), I kind of figured the cljs.jar file would have appeared somewhere by now.


I did a search in the project folder, though, and it's not there. So...what am I supposed to do? Just ignore the fact that I do not have what the main clojurescript website tells me I need? ...or just download it anyway, and if so put it where?

eccentric J00:05:07

Wait, are you following both the tutorial and the shadow-cljs tutorial?


I was going to follow the quick start guide (which I did last year on a different computer) just to refresh my memory about project folder set-up and compiling etc.,...until I was convinced by people here just to use shadow.cljs


I had barely started into the guide, just installing JDK 8 (actually adoptOpenJDK 8 ). Did I not even need to do that??

eccentric J00:05:36

JDK 8+ is required for both shadow and the tutorial. However, the tutorial is only the minimal working setup. Meaning you wont be able to connect cider (or most editors) to the REPL out of the box.

eccentric J00:05:32

Shadow-cljs is nice because it can grow with you. The defaults are good for getting your feet wet, but if you decide to use it for more serious projects you'll be able to work with it and reap benefits of what it can do. The tutorial may get to the point of writing cljs -> js but you'll likely soon outgrow it.


I'd love to know how Cognitect folks build cljs projects -- what tooling they use?

eccentric J00:05:57

That's a good question. I can't imagine working without the hot-reloading the other tools offer.


I suspect they use figwheel-main.


But I'm doubting they use shadow-cljs


Several of the high profile Cognitect folks have talked about their simple tooling workflow, which is why I wonder about their cljs workflow...


...I think I have a reasonable understanding of how they approach Clojure tooling 🙂

Alex Miller (Clojure team)01:05:38

I think it varies a lot depending on the project


I think Cognitect folks have talked quite a bit about Clojure tooling -- but very little about ClojureScript tooling, at least for the last several years.


So what I mean is that I literally have no idea how y'all build ClojureScript stuff 🙂

Alex Miller (Clojure team)01:05:40

And we work on both greenfield and existing projects so it’s not always “our” workflow


David Nolen no longer works (never has?) at Cognitect. I'd be curious what his tooling of choice is.

Alex Miller (Clojure team)01:05:36

I personally don’t know the answer to that, not really my area :)


Hum, no wait I'm getting confused with Clojure CLR ok no that's correct.


Good point re: greenfield vs existing projects.

Alex Miller (Clojure team)01:05:13

David did work at Cognitect for a number of years


Ah okay, ya that's what I felt I remembered. Its awesome that he's continuing all his hard work even now that he doesn't. Feel very blessed to have all the core contributors put in all this work so I get to have more fun programming 🙏

💯 4

I think the getting started guide might be outdated for windows. Since then, both lein and tools.deps can run on windows and pull in ClojureScript for you, and I'd assume shadow as well


@didibus There's no official Windows solution, hence the official docs must still suggest a more manual approach.


(for what it is worth, I've run the cljs quick start guide on Windows Powershell using the CLI tools in their alpha state and it is possible)

eccentric J01:05:35

Hmm I do have access to a virtual windows machine I use for gaming. I'm going to try installing shadow-cljs on it and see what I run into.


Ya that's fair. I guess lein doesn't count as official, and tools.deps powershell is alpha. My favourite on windows by the way is using the bb port

Alex Miller (Clojure team)01:05:40

And we work on both greenfield and existing projects so it’s not always “our” workflow


@jayzawrotny @didibus @seancorfield Sorry. got sidetracked there. Eccentric J, good to know that I shouldn't try to do both. I actually hadn't decided what I was going to do next yet. Still slogging through set-up it seems. Good to know there's one thing (the tutorial) I can take off my list, and I should just keep going down the shadow.cljs tutorial page. Didibus, if I remember correctly the quick start guide uses Leiningen.


It does not use lein


So can anyone explain the apparent lack of a cljs.jar file on my system (or where it might be), and what I need to do about it, if anything?


It doesn't use lein? Hmm. Well it was about a year ago I did it, so I guess after I did it I must have tried out Leiningen for some reason.

Alex Miller (Clojure team)02:05:33

Both lein and deps will cache jars under ~/.m2/repository

👍 4

I'm not using lein or deps. People recommended I use shadow.cljs instead.


(for CLJS, which I'm going to try for a while before I try Clojure)


Should I just not use the shadow.cljs guide if I'm on Windows, by the way? It seems to assume Linux.


I just tried to do the global installation as specified, and I get this:


npm install -g shadow.cljs npm ERR! code E404 npm ERR! 404 Not Found - GET - Not found npm ERR! 404 npm ERR! 404 'shadow.cljs@latest' is not in the npm registry. npm ERR! 404 You should bug the author to publish it (or use the name yourself!) npm ERR! 404 npm ERR! 404 Note that you can also install from a npm ERR! 404 tarball, folder, http url, or git url. npm ERR! A complete log of this run can be found in: npm ERR! C:\Users\[...]\AppData\Roaming\npm-cache\logs\2020-05-24T0228_55_872Z-debug.log


Actually if I go to the website I get the same error.


It was pointed out earlier that it is shadow (dash) cljs not shadow (dot) cljs


I could have sworn I copied and pasted that. My mistake! Thanks.


no worries. if you want to see a presentation about how to use shadow-cljs, a sample app, and that app deployed on netlify check out

eccentric J02:05:36

That's great!


Thanks. Reading it now.


Still wondering about that maybe missing cljs.jar file, by the way...


Shouldn't it have installed with shadow?


I'm just going to run a search on my whole drive in the meantime.

eccentric J02:05:36

shadow-cljs should handle that for you, the jar is not necessary. You should only be using the shadow-cljs CLI.


Wha..? No jar?


Did I also not need to install Java 8 then?

eccentric J02:05:08

You do need Java 8


By the way, does it really have to be specifically 8, or can it be higher (I think 14 was suggested on a couple of JDK sites).


8 or 11 are recommended. others most probably work but how many battles do you want to fight


i think i'm using 13 with no issues.

eccentric J02:05:35

11 does has a few small incompatabilities with various tools here and there and if you already have 8 installed I would just stick with it for now.


Oh I don't want to fight any extra battles. lol I just can't remember where I got the impression that the version number even mattered. Maybe a guide meantioned 8 but was written when 8 just happened to be the newest.


8 all the way. thanks.


You need at least version 8, that's all.


i thought version 10 never really worked for some reason? don't really know but i think most people recommend the long term versions which are 8 and 11


8, 11, 14 are safe choices for most Clojure stuff.


The inbetween versions are very short-lived now, the way Oracle is developing Java/JVM stuff so I personally would avoid them (and recommend beginners avoid them, in case they do cause problems).


We still have a few things on 8 but mostly we run everything on 11, although I'm developing on 14 now, mostly as a compatibility test for rolling all our systems up to 14 at some point.


Is there some kind of security argument for going with 14 instead of 8?


alrighty then. 8 it is


At least, none that I know of. 8 is still actively supported by several companies and organizations. e.g., AdoptOpenJDK.

FHE02:05:06 8 gets updated with lower-order version increments?

seancorfield02:05:41 shows update 252 is the latest version of 8 they're offering.


OK. so lower-numbered versions are not necessarily older. Got it.


Well, JDK 8 as a whole is older than 9, 10, 11, 12, 13, 14 🙂 But each major version continues to get minor updates for security and buf fixes.


@dpsutton I'm reading through your guide/presentation, and...stop me if your intended audience isn't people who have only done a thing or two in JS and CLJS, but...I am a bit unclear about a couple of things already:


1. Your very first suggested command is 'mkdir app', but I have no clue what folder that should be made in.


anywhere. you are making a new directory to work in. it doesn't make any assumptions about where it is


it can be in whatever temp directory you like or in your projects folder, or a clojure directory, or in My Documents or whatever. you are making the directory


the point of using mkdir is to show that it shows you everything you need. the directory is literally empty except for what we are about to put into it


Is it advisable to be somewhere in your HOME folder (I just created that for Doom Emacs installation, but I also started the first couple of steps of the shadow.cljs website guide there too, creating a project there.

eccentric J03:05:08

It's a common practice but could be anywhere.


i don't believe there is any advice to give here. make a new folder wherever you think it should go

FHE03:05:27 anywhere. I guess I would just make it in the same parent folder as where the top folder created by the shadow.cljs website guide project creation command gets put.


is that another project?


Looking ahead to the next steps, of creating a few files...


2. There is quite a bit of (boilerplate) text to be put in the file, but no explanations of the text. Some of it (e.g. the html) I can follow, but it would be nice to know what some of it means.


More importantly, though...


(this is a lightning talk to get you up and running in 5 minutes. everything should seem sensible and is explained in the users guide)


3. After the "Create shadow-cljs.edn" step" there is a "Index.html" step. Should that be "Create Index.html"? ...and according to the little tree diagram should that be in a subfolder of the 'app' folder called 'public'?


If you're open to suggestions (on how to get through to people at my level :P), I would change the titles for the files to be created to all be "Create <filename.ext> in:


" with a tree below.


Would be clearer. Only saying that because i noticed you requested suggestions. 🙂


this presentation was originally given accompanying a talk where i could flush out details. owing to this i had to keep the slides simple and visible from a distance so lots of context was supplied from speaking


Sure, but the amount of context in the actual text of the presentation seems to change over the course of the list. 😉


2nd file no 'create'. 3rd file also no tree. 4th file...actually I can't really tell if there's a 4th file. Does the 'Closer look' section mean 'Create main.cljs in \app\src\app\'?


what is the third file with no tree?


closer look and boiler plate are both the same file


just had to break it into two slides


you can reference the repo for checks of what is going on


The 3rd file with no tree was the Hiccup section. I just saw your answer in the other thead, about how that section is not about a file to creat at all.


"closer look and boiler plate are both the same file" Ahhh. OK. and now I see it's all together in the Source files section.


...but that section is just one source file, yes?


yes. i'll link it again here so we can stay in a single thread


Oh wow. There's a lot more in that main.cljs file than the presentation/guide contains. OK.


its on slide 16


So...running 'npm i react-modal' makes the middle part of the main.cljs file appear??


no. npm i react-modal installs the javascript dependency. then you edit the file to use it as shown on slide 16


i'm putting the new source and leaving to you to see that ["react-modal" :as Modal] is new along with the other lines


I think it's coming together for me.. It's a bit confusing that you repeat some but not all of the previous contents of the main.cljs file.


Yes I’m trying to focus on the diff


But go see the complete file on github to see what it all should look like


I think if you included the whole thing on that step of the guide (that slide), including the part at the bottom that you refer to as 'boiler plate' earlier, it would be a lot easier to see what's going on.


Or at least replace the boilerplate part and the beginning part with [...]


Beginning part:

(ns app.main
  (:require [reagent.core :as r]
            ["react-modal" :as Modal]))

(.setAppElement Modal "#app")


While server is running you can just install

npm i react-modal
then modify the defn app part of main.cljs
(defn app
  (let [modal-state (r/atom false)]
    (fn []
      [:div {:style {:margin "auto"
                     :margin-top "100px"
                     :width "600px"}}
       [:h1 "hi"]
       [:button {:on-click #(swap! modal-state not)}
       [:> Modal {:isOpen @modal-state
                  :onRequestClose #(reset! modal-state false)
                  :contentLabel "Example Modal"
                  :shouldCloseOnOverlayClick true
                  :style {:content {:top         "50%"
                                    :left        "50%"
                                    :right       "auto"
                                    :bottom      "auto"
                                    :marginRight "-50%"
                                    :transform   "translate(-50%, -50%)"}}}
         "This is a modal"
          [:li "With content"]
          [:li "And lists"]]]]])))


Again use the slides to see what is new and then check github for the whole file


That looks correct


You have cleared that part up for me, thanks. Just trying to help you tweak your guide, if you want, to reach noob-whisperer level. ;)


It’s a balancing act. If I put too much on one slide it gets cut off. And spliting over two slides can be confusing as seen early on

eccentric J03:05:47

The other nuance is how many people are coming in with no prior dev experience, some prior dev experience, or a lot of dev experience. Currently, I don't know.


Yep. I'm just suggesting that for Slide 16, the text "and change the defn app part of the main.cljs file" might be more valuable than repeating the

(ns app.main
(:require [reagent.core :as r]
["react-modal" :as Modal]))
part. Less text overall to boot.


Same question for the 'Hiccup' section, except it doesn't even have a file extension or a tree diagram to go by.


that's explaining the concept of hiccup. the next slide shows the actual hiccup to use and where to save it


Ohhhh. So...there's a file with all html (index.html) and also a main.cljs with some hiccup in place of html sprinkled in it?


Hey, I'm actually going to try walking through that guide of yours now, @dpsutton. Thanks for the clarifications in the sidebar. Quick ultra-basic question (for anyone) before I start: Is there any issue with the top folder name (project name?) containing underscores or spaces? I just have noticed a-lot-of-hyphens-everywhere-in-project-names-in-multiple-online-guides.


The top folder is independent of the contents. Spaces in directory and file names seem almost always wrong to me


I mostly live in Windows at the moment, but having used Linux a bunch of times I typically use underscores all over the place instead of spaces in file and folder names (in Windows). I just remember Linux having trouble with spaces.


So...not actually asking if anything would choke on spaces, but actually that question about underscores.


The top level directory is independent of the project. It can be named whatever you want


Source directories and files need to use underscores due to requirements/limitations of the jvm


underscores or hyphens, right?


Underscores. Use hyphens in the namespace names but the files need underscores


😕 Of course it's different. OK thanks.


I am extremely unclear about the term 'namespace' by the way.


I've come across it so much during all the CLJS videos I've watched, but never with any explanation. Is it just variable scope, like within-procedure vs global?


read the reference first


That ref is hard to understand. I guess I'll wait until the topic comes up more organically during something I'm doing.


For now my guess is just that namespaces prevent two variables of the same name (maybe one in a library and one in a program using it) from being accessed at the wrong times instead of the other one.


@factorhengineering Every Clojure function, etc. is created within a namespace. The built-in Clojure functions are mostly defined in the namespace clojure.core. Some additional string-related ones are in a namespace called clojure.string. Typically when you start a library or application, you create one or more namespaces of your choice, (not clojure.core or anything else used by an existing library you want to use, but it is easy to avoid conflicts), and create your code in them.


And yes, the purpose of namespaces is to let you use whatever names you want within your own namespace, and not conflict with the same names chosen in the namespaces created by others.


Not everyone does this, but one way to ensure uniqueness of namespacee names is to take a DNS name like that you or the organization you work for owns, and use namespace names that begin with com.mydomain, e.g. com.mydomain.mynewlibrary


@dpsutton One more dumb question about your guide: When you say this file is in this folder:

└── app
    └── main.cljs
I know you mean \app\src\app\main.cljs because you named your top level folder (and thus your project?) 'app', but...


you don't know that


is the 2nd 'app' in the path meant to always be exactly 'app' in any project, or is it meant to be a repeat of the project name (the top level folder name)?


it could be FHE/src/app/main.cljs


I mean if I were following your guide exactly, i.e. with mkdir app


mkdir app


but this is an important concept called the classpath. by default src is the classpath root. clojure will look for the namespace app.main in a folder called app and a file called main.cljs from the classpath roots


In your case it's ...\app\src\app\main.cljs


so for a file to have the namespace app.main, it needs to be at src/app/main.cljs


so the first app is irrelevant. if that makes sense


but if the top level folder were foo, would it be \foo\src\app\main.cljs or \foo\src\foo\main.cljs ?


That second one would have a namespace of foo.main


Ah. I think you answered it above. it's the 1st one


yeah. the namespace and the hierarchy go hand in hand


"clojure will look for the namespace app.main" Got it.


and namespaces need hypens but filenames get underscores is the one place they diverge


almost always trips people up at least once


and just to be clear, I think I heard that namespaces CANNOT have underscores. It is also true that filenames CANNOT have hyphens, or can those have both?


I have to say, the namespace reference is terrible if you're trying to learn Clojure -- it talks about vars and mappings but it says nothing about project structure and how that relates to namespaces.


Namespaces can have underscores, as can file names. hyphens in Clojure namespace names are searched for in files that have hyphens replaced with underscores.


thanks @andy.fingerhut. I just tested it and namespaces can indeed have underscores but i've never seen anyone do this


It is unusual, certainly, but not illegal


(I've seen beginners do it "by accident" because they get confused about the rules)


Just to cover file names and folder names, is this correct: • file names can have underscores but not hyphens • folder names can have underscores but not hyphens • namespaces can have hyphens but not underscores (and really the hyphens and just replacements for underscores in the corresponding file names and folder names)


namespaces can have either


file names that have hyphens will not be found by Clojure when require'ing namespaces.


But don't namespaces essentially refer back to folder names and file names?


The file system will let you create such file names, of course, it just isn't productive for Clojure development 🙂


well yes. lol. just talking clojure here


This is maybe unnecessary info for you at this point, but you can create namespaces dynamically in Clojure that have no files on the file system to store them.


Namespace foo-bar.quux => filename <classpath>/foo_bar/quux.clj (or .cljc or .cljs


Don't worry about that too much right now, if it is overload of detail


(where <classpath> is going to be your src folder or your test folder typically -- for dev/test they're typically both considered, for production just the src folder, depending on how you set up your project)


^ What Sean said


@seancorfieldThat's what I was thinking. Good to see it in an example.


By the way, are the top level folder name and the project name synonymous. I'm guessing yes, but still...


i've never made a dynamic namespace. nor worked in a code base where it happened


not necessarily


notably, you can clone any repo into whatever foldername you want


for instance, git clone [email protected]:clojure/core.match.git custom-name will clone the core.match library as custom-name. The project will behave just as if it were saved into a directory named core.match


does top-level-folder-name = project-name ?


No, not necessarily.


no. as demonstrated above


OK, but usually?


unless you take specific steps to break that?


The "project name" comes into play if you are publishing a library to Clojars/Maven (a repository). Those have a group ID (like your company name or your GitHub username) and an artifact ID (the actual project name). e.g., fhe/my-project, seancorfield/clj-new. That doesn't have to correspond to anything in the source code but it will be part of the descriptor that is used to deploy the project to a repository. It's somewhat common to have the namespace structure of the project match the group/artifact tho'. So the fhe/my-project project would be in a top-level folder called my-project in which would be src and test folders (typically) and inside src you'd have fhe/my_project.clj with a namespace -- does that help?


If you have the :new alias in your ~/.clojure/deps.edn file pointing to the clj-new library, you can see this in play by doing:

$ clojure -A:new app fhe/my-project
That will create my-project/src/fhe/my_project.clj containing (ns


Ehh...maybe? Actually I don't get this part: inside src you'd have fhe/my_project.clj


Do you mean inside src you'd have fhe/my_project/src/my_project.clj


dan@dan-mbp /tmp % clojure -A:new app fhe/my-project
Generating a project called my-project based on the 'app' template.
dan@dan-mbp /tmp % tree my-project
├── deps.edn
├── doc
│   └──
├── pom.xml
├── resources
├── src
│   └── fhe
│       └── my_project.clj
└── test
    └── fhe
        └── my_project_test.clj

6 directories, 8 files
dan@dan-mbp /tmp %


no. I mean exactly what I say. Try the command above.


Ugh. This thing will not let me edit for some reason.


Trying to edit my post is flash-scrolling me hours back in the chat for some reason.


What I mean was:


See this:

seanc@DESKTOP-QU2UJ1N:~/clojure$ mkdir temp
seanc@DESKTOP-QU2UJ1N:~/clojure$ cd temp
seanc@DESKTOP-QU2UJ1N:~/clojure/temp$ clojure -A:new app fhe/my-project
Generating a project called my-project based on the 'app' template.
seanc@DESKTOP-QU2UJ1N:~/clojure/temp$ tree my-project/
├── deps.edn
├── doc
│   └──
├── pom.xml
├── resources
├── src
│   └── fhe
│       └── my_project.clj
└── test
    └── fhe
        └── my_project_test.clj

6 directories, 8 files



seanc@DESKTOP-QU2UJ1N:~/clojure/temp$ cat my-project/src/fhe/my_project.clj

(defn -main
  "I don't do a whole lot ... yet."
  [& args]
  (println "Hello, World!"))


Actually I don't get this part: inside src you'd have fhe/my_project.clj Do you mean inside src you'd have my_project.clj ?


Is see the big tree you posted. So why is it repeating the top folder name in the subfolder of src??


It's a common convention, that's all.


Go back and read my explanation, then read the commands I showed and their output. Then go try that yourself.


Also, didn't we just talk above about the fact that the top folder name doesn't matter and is not repeated and clojure looks for app.main (meaning /app/main.cljs)?


I'm getting more confused now,.


Right, it is not related. It's just common practice for them to match.


Again, read what I said about project names vs namespace structure above.


The top-level folder name, the project "name" (group/artifact) and the namespace structure are all independent -- but it is common practice to connect them like this.


What I gather from that was that you could decouple top folder name from project name if you really wanted to.


clojure -A:new app fhe/my-project


Only the namespace structure and the file path structure (in src and test) are tied together.


I don't get the line above btw.


Why would you name your app a name with a slash in the middle??


Go read the clj-new project README where that is explained.


Libraries -- on Maven/Clojars -- have a group ID and an artifact ID and they are separated by a /


For example, org.clojure/clojure -- group ID org.clojure (a reverse domain name for ) and artifact ID clojure ... and org.clojure has several projects: clojure, clojure.string, java.jdbc etc.


@factorhengineering I have never seen a project or library name with a slash in the middle. Is there something specific that someone else said that leads you to believe they were recommending that?


@andy.fingerhut he's misreading the example I gave.


Yes, in seancorfield's example, he has:


clojure -A:new app fhe/my-project


So that would be group ID fhe and artifact ID my-project (by convention)


Why not just

clojure -A:new app my-project


That is explained the in README for clj-new. Please go and read it.


OK, but if one is not trying to produce a library, and is just working on smaller thing, that slash and 'group ID' would not be needed I guess.


It's a good practice to get into.


I have not even heard of clj-new before. I think it's down the reading list for me. 😕 Right now I'm just trying to correctly follow @dpsutton’s guide.


If your project name, top-level folder, and namespaces follow that convention, your code is less likely to conflict with code from another library.


THAT statement made complete sense to me.


I thought you had talked about clj-new the other day -- when you were asking about deps.edn and creating new projects using that...


If lots of people are uploading libraries there could be conflicts unless the namespaces are unique, so the convention is to have a unique identifier PART of the name (assigned to people or organizations) and a more descriptive PART of the name. That would make sense to me.


Libraries have group IDs and artifact IDs -- those must be unique.


so the 2nd part has to be unique too? That makes less sense to me


The combination must be unique.


If those are already unique why even have group IDs


OK, so the artifact IDs are NOT, on their own, unique. all good. ok


The group ID + artifact ID together must be unique. They are the "coordinates" of the library.


group-id/artifact-id -- in Leiningen's project.clj you see [group-id/artifact-id "1.2.3"] for dependencies... and in Clojure CLI / deps.edn it is written group-id/artifact-id {:mvn/version "1.2.3"}


As for whether you mentioned clj-new to me way back, if you did I didn't get that message. All I remember about deps..whatever it's called is that it was a built-in thing that does some of what Leiningen or shadow.cljs does.


Sean can correct me if I have this wrong, but I'm pretty sure that the group ID + artifact ID naming scheme was originated by Maven, a tool/system for publishing and distributing Java libraries, that Clojure/Java does not try to replace, but to build on, because it is common to want to use existing published Java libraries in Clojure/Java programs.


Inside a library you can have any namespaces -- but again, you want your code to not conflict with someone else's code so following group-id.artifact-id in general is a useful convention (for a file in src/group_id/artitact_id.clj)


Right, the two conventions are separate but essentially overlaid in order to keep code from conflicting, both at the library artifact level (group/artifact) and in the Clojure source (`group.artifact`).


OK. Makes sense. Thank you.


It would be... somewhat surprising... if you depended on a library as [foo/bar "1.2.3"] and then end up having (:require [quux.wibble]) to bring in its code... library names and namespaces are not usually that disconnected.


Now I feel like I need to pull myself out of this rabbit hole. lol. I mean I'm sure I'll use the info, just not right this second. i want to get through and maybe now and then finally actually make something.


I'm crying to actually write some of my own code at this point. lol. So...much...set-up.


I think it's important to understand the project basics (for Clojure) in order not to get lost in ClojureScript's ecosystem...


It's why I always recommend beginners learn basic Clojure stuff first and then move on to ClojureScript


(It's why I also recommend not trying to learn both Clojure and Emacs at the same time)


Otherwise you can get rather overwhelmed... as I think you have at times?


So many things to install, so many things to figure out about how to install things, and even with the tools installed this setting up a project or whatever is needed to actually get a cljs file for me to edit in front of me is even more confusing.


Yes, there are a lot of moving parts.


Clojure isn't "easy".


I understand your Emacs comment, but that bit of learning I willingly signed up for, having tried Emacs for a bit way back. At least it doesn't feel like pandora's box like this set-up stuff does.


Hmm...I guess I mean a Matroska doll


Actually, what I really mean is a hydra. lol


Emacs is all of those and more 🙂


I'm at peace with the file opening and saving and weird versions of copying and pasting in it now. I can just use it for that (and CIDER when I figure out what that exactly is) for the moment.


I first started using Emacs back in its 17.x and 18.x days... then moved on to other editors and when I learned Clojure (ten years ago) I was using TextMate! After a while I came back to Emacs (23.x I think) for a few years, then switched to Atom in 2015 and that's what I use now.


...and put off learning vim for the moment as well!


My knowledge of vim is pretty minimal. I know barely enough for basic editing.


All that investment in Emacs AND you do Clojure and STILL you moved away from it? That's sad to hear, from the perspective of someone who tried hard to identify the best programming language and editor to learn and picked Emacs as the editor. lol


About 80-90% of Clojure programmers used Emacs a decade ago when I first picked up Clojure -- but that percentage has dropped slowly each year since I think... I'd have to check the annual survey results...


It’s 45/45/10 or so emacs cursive and now vs code


Emacs was down to 45% in 2020.


But those are of course market share and hopefully the entire clojure user base got bigger :)


The 2010 survey suggested about 70% Emacs usage so I was slightly off above.


Oh well. I have tried org-mode and like it a lot. Good tie-breaker.


I never got into org-mode but I know people who absolutely love it.


Org mode can do things few other editors can. Org mode grew organically over time. A new markup/editor will need significant investment before it can reach feature parity.


It really just isn't a use case I seem to need. And when I've tried to use it in the past, I just get frustrated with how it works.


Yeah, that is fare. For me, there are many many useful features, like org capture which let's you capture a thought from any buffer. Deft let's you search all your notes. The table editor is just auto magical. And I also sometimes build presentations in org mode (by exporting to reveal.js).


Since I am already comfortable with Emacs, I don't need to learn "n" different tools. It is a huge time saver.


Yup, totally get that. Emacs is really an operating system that just happens to let you edit files 🙂 I never really got comfortable with Emacs, even using it over decades. I never found a setup I really enjoyed.


I can suggest Doom Emacs. I find both Spacemacs and Doom enjoyable. They both come with sane configurations. These are recent projects and maybe you have not tried these.


I do see value in an editor that works in a bare terminal but I rarely need that low-level stuff these days -- I can connect a REPL to any server, any process; I can connect Atom to that too. I can do everything from Clojure, in my editor, But when I'm not "programming" I like modern UI/UX tools -- so much so that my non-programming activities are all done in Windows or MS apps on my iPhone, my Windows laptop, and even my Mac desktop (which runs Windows 10)...


Not interested. I have a lot of REBL integration invested in Chlorine now and use bare socket REPL everywhere. I don't want anything to do with CIDER/nREPL.


Yeah, once you are already invested in something, there needs to be a solid reason to switch.


I totally get that.


And I love REBL 🙂

eccentric J05:05:51

I get the love but thankfully my favorite Notebook app, Notion, just became free for personal use.


Emacs dropped slightly to 43%, IntelliJ/Cursive rose slightly to 32%, and VS Code with Calva had the biggest increase to 10%.


Yeah, Atom is still very much a minority choice but I really do like it. And I love that the latest version of Chlorine (for Atom) lets me customize it via ClojureScript (using sci so there's no compile step and changes apply immediately).

eccentric J05:05:02

I used Atom + proto repl for a year or so before Spacemacs clicked and became my favorite. It was pretty good overall and Chlorine seems even better from what I’ve seen.


Yup, I initially used ProtoREPL and loved it. I was disappointed when Jason moved on. I was VERY excited when Chlorine appeared and I love the inline eval results (just like ProtoREPL) and the socket REPL integration -- so I can connect Atom/Chlorine to any process locally or remotely (since socket REPL is built-in to Clojure and just JVM opts are needed to start a socket REPL on any process).


Chlorine recently introduced interactive rendering (basically Hiccup and ClojureScript to power React components that render inline in the editor!).

🤯 8

And then the full customization via ClojureScript -- so all my REBL integration is done that way now.


@jayzawrotny I was looking hard at SpaceMacs until at the 11th hour Icame across a bunch of comments that Doom was similar but much faster.


Do you use EVil? Itried vim for a couple of days a year ago. Even more of a mindmelt than Emacs but maybe worth it in the end.

FHE05:05:52 Dvorak typing which I'm very glad I switched to years and years ago.

eccentric J05:05:23

Yes, I use Evil mode. I learned vim 10 years ago and always enjoyed editing that way. That's true DOOM is probably more performant but I think Spacemacs is more batteries included and a bit easier to use. Though from what I understand using DOOM forces you to learn more about how to work with Emacs' internals so that's not a bad thing.


What aspects of spacemacs are more 'batteries-included'? Really interested, since I'm just barely starting with Doom and could possibly switch.

eccentric J05:05:06

Comes with fuzzy search, modern UI features, integrates auto-complete and helm (searchable menu system) into everything and there’s a strong consistency with how everything is accessed. The biggest add are the layers, which are groups of emacs packages that solve bigger problems. For instance there’s a Clojure layer that includes cider, clojure syntax, clojure key bindings, and clojure formatting


I'm not very familiar with the other things on your list, but I do know Doom has the ones I recognize: helm (or maybe ivy which is like helm?) and...i'm not sure if they're called layers in doom, but I think it turned on a bunch of clojure things when I went to clojure...mode? (mode might not be the right word)

eccentric J06:05:34

That’s good to know. Clojure-mode is the right word but it’s only one package I think, the spacemacs layer includes clojure mode among a few others.

eccentric J07:05:46

Ah Doom has modules instead of layers. It’s less polished in the UI\UX. It’s less streamlined so it may take more config to make it the way you like. Spacemacs is more about reasonable defaults and is more like a complete framework that don’t want to get stuck customizing Emacs too much.

eccentric J07:05:09

But Doom is more about customizing the experience to your liking


While Doom makes customization much easier as compared to Spacemacs, it works out of the box too.


All the benefits you mention about Spacemacs is there in Doom too. The Clojure module includes CIDER, clj-refactor, key bindings.

👍 4

Is there any way to export chat logs from this? I'm trying to get back and look at the answers about filename underscores (yes I'm already fuzzy on the details!), but this interface keeps unloading older chat messages. I hate that. It's all or mostly just text! Keep it all in memory!


All the chat here is mirrored to Zulip and also to the ClojureVerse log... I'll get you the URLs...


So...Zulip is easier (possible) to export from compared to Slack?


Is it better? Are you on that too? I don't really want to sign up for yet another thing. lol


Yes, I'm on Zulip. It's much lower traffic than Slack. But it has unlimited archive/search.


Many channels here are archived to Zulip (it's one-way: what you post on Zulip does not appear here)


Some channels are mirrored to Matrix/Riot as well but I think that's more focused on mirroring IRC which still has hundreds of active users.


(and there's a Discord community, and Telegram and various others 🙂 )


FWIW, I login to Zulip using GitHub so it's not like I had to signup for anything new really.


Thanks for the log link. Much more searchable (though lists and probably other things are messed up by the logging.).


also the IRC tip. I have HexChat open already just to get general Emacs help. Didn't know I more clojure goodness was just a /join away.


Note sure if @dpsutton is still here, but I just ran this and got this, which doesn't lookequite right:


npm i react create-react-class react-dom shadow-cljs npm WARN deprecated [email protected]: core-js@<3 is no longer maintained and not recommended for usage due to the number of issues. Please, upgrade your dependencies to the actual version of core-js@3. npm notice created a lockfile as package-lock.json. You should commit this file. npm WARN [email protected] No description npm WARN [email protected] No repository field. + [email protected] + [email protected] + [email protected] + [email protected] added 119 packages from 191 contributors and audited 119 packages in 10.867s 1 package is looking for funding run npm fund for details found 0 vulnerabilities


There’s a warning but I’ve been ignoring it. The other warnings are warnings about your own package.json file. About no description and repository information. Completely benign

Fredrik Meyer11:05:13

I have a beginner concurrency question. I'd like to update a Java array faster. Right now I do this:

(doseq [i (range (count pxs))]
        (aset-int pxs i (nth pxs-sorted i))
        (when (= (rem i 50000) 0)
          (println i)))
      (println "done setting pixels")
But it takes several minutes (the array has length 360.000). My idea was to have several threads do it at the same time, but as I'm new to Clojure concurrency, I'm not really sure where to get started. Ideally, I would like to have the content of the doseq be done with many threads, but resuming the program after the loop is done being processed.

code star 812:05:47

Does pxs-sorted rely on accessing the entire array of pixels? Or can you break up the array into separate sub arrays so you could process it in parallel?


I dunno what pxs-sorted is, but nth is probably doing a linear scan on it.


other micro optimizations: use dotimes to avoid allocating a seq, use zero? instead of (= … 0)


re: parallelization: that’s risky with a backing array, but pmap is an easy thing to try.

code star 812:05:18

Ya. Probably linear, and needs to load entire array to sort it.

code star 812:05:52

Yep. I was thinking pmap to parallelize. But probably can't because of the sort algorithm.

code star 812:05:12

Divide and conquer, parallelize

code star 812:05:29

O(Log n) complexity instead of linear O(n)

Fredrik Meyer12:05:43

Oh, that "nth" is linear, I didn't think of! I guess get is better for arrays? Actually, I tried pmap now. It helps quite a lot. The computation took ~2 minutes instead of 7 minutes, with correct result. (though I still think 2 minutes is quite long for just putting values inside an array...)

code star 812:05:22

I'm super newb at concurrency. It's interesting... What other ways could you try? Agents or futures?

Fredrik Meyer12:05:10

Hmm, I get nil when trying to use get instead of nth ... I think its because seqs dont have indexes maybe?

Fredrik Meyer12:05:10

Ah!!! That was it. Doing a combination of pmap and to-array in the beginning got the time down to <1 sec

code star 812:05:19

Can you post pxs-sorted body?

code star 812:05:51

Because I'm actually surprised pmap worked on a sort

Fredrik Meyer12:05:29

The reason it works is that pxs-sorted is not changed. All I want to do is to copy the contents of pxs-sorted into pxs in the same order, in place, which was kinda clunky in clojure...

(let [pxs (q/pixels)
          pxs-sorted (to-array (sort-by q/brightness pxs))]
       (doall (pmap (fn [i]
                      (aset-int pxs i (get pxs-sorted i))
                      ) (range (count pxs)))))
      (println "done setting pixels")

Fredrik Meyer12:05:04

Oh! Just noticed the pmap is slower! So actually, my biggest mistake was using nth...

code star 812:05:38

Ahh. So it's not actually using a sorting algo. Gotcha.

code star 812:05:03

Clojure is such an elegant lang.


if it’s an array, aget is better


but i dunno why you need to array that – it’s immutable anyways


arrays help for various reasons, but chief among them is fast mutability

Fredrik Meyer06:05:46

@U07S8JGF7 I think I had to do it because sort-by returns a seq and they don't have indexes. Unless I have misunderstood something, of course.


@UA277415W Vectors have constant-time, indexed access. If instead you do (vec (sort-by q/brightness pxs)) you’ll get roughly the same lookup performance.

David Pham16:05:06

I would be interested in training my skills with clojure.test.check, and I have trouble to train my skills at writing properties. I read Clojure Applied from Alex Miller, watch the talks from the conj and from John Hughes at Clojure West and the latest one about pure function (how to specify it). Does anyone know if there is a repository (or something else) where there would be some faulty code and we would use test.check to write properties to find all the possible bugs?


First, I would say that if you are thinking of property based testing as "a way that if done properly, can guarantee finding all of the possible bugs", then I would mentally take a step back from that. Why? Because it is still a form of testing, albeit one where a myriad of test cases you might never think to write will be automatically generated for you.

💯 8

Testing can find the presence of bugs. It will not prove the absence of bugs.


I think it is an amazingly good testing technique to use, and can greatly increase your confidence in how well tested the code is.


But as a very simple example, suppose you were testing a function to sort a vector of numbers in increasing order, and your postcondition checking code verified that the returned value was a vector that was the same length as the input vector, and did no other checks. You could use that as a test on a function that simply returned the input vector, and the automatic generation could generate thousands of test cases for you, and it will never find the obvious bug that the returned vectors are not sorted.


Determining what properties to check is part of the art of property-based testing. How complete/thorough do you want to be there?

David Pham19:05:32

This is exactly the kind of experience and sensibilities I want to acquire. I totally agree that it is not a way to find all possible bugs, but more a way for verifying assumptions I have about my code (and also checking when I update my deps).

David Pham19:05:05

And yes, knowing which properties to test is also what I would like to learn 🙂

David Pham19:05:01

I had in mind, there should be some projects/repository with intentional bugs and the goal would be to find them thanks to either PBT.

David Pham19:05:52

(As for for sorted collection, it is a textbook example of recursive properties, but I would like to learn the tool more than having a unrationnal desire to apply it).


If there is such a learning resource, I have not heard of it, but yes that would be nice

David Pham20:05:13

I think it would be a good resource honestly. Learning to debug probably belongs to the most underrated skills.


i’m writing a macro that I want to accept a quoted list, e.g. '[Foo] , but because the macro is receiving the reader form (quote [Foo]) i’m iterating over it incorrectly. What would be right way to grab the list? I tried unquote but it gives me unbound function error, so doing this and it works (if (= (first tags) 'quote) (second tags) tags)


@alidcastano That is correct. ' is just a reader shorthand for (quote ,,,) so the form is actually read in as (quote [Foo]) -- macros do not evaluate their arguments.


You could just pass [Foo] and your macro would not evaluate it anyway.


the linter gives me an unresolved symbol error, which is why I’m quoting it. so I’ll just stick with grabbing the desired form (if (= (first tags) 'quote) (second tags) tags)


You could pass '[Foo] to a function and it would do what you want.


Or you could tell the linter (`clj-kondo`?) about your macro so it doesn't flag the unresolved symbol. Ask in #clj-kondo for more information about that.


This is certainly an area where you know more than the longer. You are creating syntax so the linter is irrelevant here.


(the first rule of Macro Club is Don't Write Macros -- so be sure that what you're trying to do can't already be written as a function instead of a macro)

🙂 16
Kazuki Yokoyama20:05:17

Hi, looking for an advice: should I use clojure.spec or stick with plumatic/schema on new projects?

seancorfield20:05:17 Personally, I'd recommend clojure.spec -- it is useful for a lot more things, in my opinion, and it's also "official" since it's from the core team.

David Pham20:05:42

Is spec2 compatible with specs.alpha?


We use it heavily at work and I wrote up the various different ways we use it


I noticed that you cite “Deriving code from specs” in your post. how is the inspectability of spec? do you rely on stuff like (= (:pred s) int?) ?


I assumed that since spec is predicative and functions in clojure are fairly opaque, that the specs themselves would be fairly opaque


You can get the form of a Spec and then you can walk the form, yes.


For example, we have s/keys specs for database records and from the form of those specs we derive a series of CRUD functions automatically.

👍 4

We use a similar technique to turn explain-data into user-facing error messages, based on partial descriptions of predicates that may fail in the Spec. For example, knowing that contains? is used to detect required keys in hash maps, lets you identify missing parameters/keys in maps automatically from the spec failure.


I guess I was just curious how you handle using a spec like

(s/def ::error int?)


if int? is a function, all you can do is call it (not super useful if you don’t know what it is) or test for equality


so I guess you have to rely specs using predicates that you have access to


so in this case, (s/def ::error #(int? %)) would be far less useful from the perspective of using specs to generate code?


It's still just code-as-data -- you can walk into it to get at the predicate

Clojure 1.10.1
user=> (require '[clojure.spec.alpha :as s])
user=> (s/def ::error1 int?)
user=> (s/def ::error2 #(int? %))
user=> (s/form ::error1)
user=> (s/form ::error2)
(clojure.core/fn [%] (clojure.core/int? %))


We look for clojure.core/fn in the first slot of a sequential form and then take the last form out of it and inspect that.


:thumbsup: good to know


"It's just data" 🙂


We actually have a function that walks through functions and expressions to find the most likely predicate and its arguments, e.g., #(contains? % :foo) would return [clojure.core/contains? :foo] as a tuple. It's "accurate enough" for the heuristics we need to turn spec failures into predicate tuples that we then use as keys in maps to messages (we try the full tuple first, then drop arguments from the tuples, finally trying just the predicate itself to find a match).

notbad 4

Spec 2 isn't directly compatible with clojure.spec.alpha (aka Spec 1) but Spec 2 isn't ready for use yet


@seancorfield - thanks. This is a useful article


(ns me.specs-playground
  (:require [clojure.spec.alpha :as s]))

(s/fdef print-digit
  :args (s/cat :just-an-int int?))
(defn print-digit
  (println (str just-an-int)))

(defn -main
  [& _]
  (print-digit "asds"))
I'm playing with specs right now, and here ^ I have simple example where I added specs for print-digit function. Is there a linter which would catch that I'm calling print-digit with a wrong argument (String instead of intiger)? Or specs are only useful for writing tests at the moment?


@prnsml argument-checking is enabled by calling instrument

Kazuki Yokoyama20:05:34

Thank you, @seancorfield. And should new projects use Spec 1 or Spec 2 (although it is considered not ready for production yet)?


Spec 2 is still changing a lot. It's not really ready for any usage yet. Unless you are deliberately exploring/testing it to provide feedback to the core team 🙂

👍 4

Anyone got any experience of CORS for CLJS using cljs.ajax to a http-kit server


what libraries I should be looking at, or just anything I can read at all to help


Question, I’m trying out several JSON packages, however only with org.clojure/data.json decoding i can easily translate datetime string to actual DateTime


what is the idiom to do this in Clojure for other packages?


should I first just import the full data set and then do updates in the data set after the fact?


example being cheshire*


@patrick.glind Each library will have its own mechanism for controlling date conversions.


the weird thing is, I started a new clojure app to try out all kinds of JSON libraries


however don’t really see the option to convert decoding json data to the right ‘type’


with clojure/data.json I can do this:

(json/read-str json-file
:value-fn json-value-reader ; each key/value will go through this function :key-fn keyword)


but for cheshire and jsonista


i don’t see that kind of feature


JSON doesn't support dates as a datatype


You need to use your own scheme on top


yeah, it comes in as a String


which is fine


i just want to immediately convert it to DateTime


same as I would do with a CSV file


but then i figured, like with data.json, there should be an idiomatic way


to convert on import


The problem is that there isn't a objectively "correct" way of knowing what date/time type a JSON string represents. For a start, there are several different ways of representing dates in the JVM.


According to the Cheshire readme, the decode function can take a custom decoder that handles custom parsing based on the field name


In 2.0.4 and up, Cheshire allows passing in a function to specify what kind of types to return, like so:

;; In this example a function that checks for a certain key
(decode "{\"myarray\":[2,3,3,2],\"myset\":[1,2,2,1]}" true
        (fn [field-name]
          (if (= field-name "myset")
;; => {:myarray [2 3 3 2], :myset #{1 2}}


I don't know if it changed since that ticket.


@didibus, what is the clojure idiomatic way to convert it after import?


@seancorfield, yeah I saw that in the doc’s


[key-fn array-coerce-fn]


i tested to see if i could make it work, but no luck


Oh, I missed that part. Sorry.


i totally understand if the packages don’t convert the data, just do the import part


i’m just curious how to process it afterwards idiomatically


Hum.. are you, actually reading the doc, are you sure it's not for map?


Is this json you get from elsewhere? Or you encode it yourself?


i get it from elsewere


parsing is no problem with all 3 libraries i’m working with


The best would be to encode the date to a map with a custom encoder. And then I think you can use that custom decoder function to decode it back to date


From elsewhere, hum, so how can you tell the difference between a string with a date and just a string of text?


it’s just that so far, only data.json supports converting types on import


the key is :created_at (i convert to keyword)


In any case, you can post process it with clojure.walk


"created_at": "2018-04-24T14:51:11.220-04:00"


Oh I see, then what Sean said from the doc, you sure it doesn't work?


i’m able to convert it with this:


(defn json-value-reader [key value]
  (if (= key :created_at)
    ;; (java.sql.Date/valueOf value)
    ;; (java.time.LocalDate/parse value)
    ;; (read-string (str "#inst" value))
    (java.time.ZonedDateTime/parse value) ; 


for data.json


The Cheshire readme is confusing -- decode is just an alias for parse-string and that only works for arrays.


I would just stick to then.


yeah, i was getting to that point


but was hoping since i’m a beginner, it would be me that’s missing something haha


my next thought was: after import: just do a mass ‘update’ on the data, convert the string->ZonedDateTime


but wanted to find the idiomatic Clojure way


I think that's your best bet


You can do a clojure.walk post processing pass I guess


i’ve never worked with that


Unless we're missing an obvious feature in Cheshire


it’s the same with jsonista


for decode there are very few options


As a beginner, it's reasonable to use the org.clojure libs as a first step if they do what you need. Otherwise you can get overwhelmed by all the choice (Clojure has so many libraries for any given problem).


thanks guys


the weird thing is, i was trying to use the org.clojure library for parsing a ~30mb XML file


however that caused a lot of performance trouble ha


Ah, and it was very slow?


I'm not suggesting using it (definitely stick with for now) but something I've used before with some success for this sort of thing for more complex data structures is Metosin's spec-driven transformations


org.clojure for json works fast enought for me 🙂


Yeah, well, you're going to trade off performance for functionality in a lot of situations I suspect 🙂


haha true true


btw @seancorfield, i liked your post on next.jdbc, next time i work with a database i will definitly try your library!


I feel we are missing something. Kind of surprised Cheshire and jsonista wouldn't support custom decoder/encoders at this point. Doing a post processing pass seems way slower, and they're all about performance


i was thinking the same thing, confusing..


Probably because they're both based on Jackson, a Java library, and that doesn't expose the feature -- or they're both so focused on performance that they don't want arbitrary value transformations?


i will try some more things


thanks for your help


1 thing that is very confusing to me as a beginner though is how the clojure.xml library is having terrible performance for me


like i said, it’s a ~30mb file


and i can read it, run parse on it


When you recur a function you're not creating a new thread, so a with-local-vars binding will persist through recursive calls correct?


I mean, to be fair, that's pretty big :p


Lots of editors would choke on that size of file


haha, alternatively I found BaseX which parses it in split second and I can run XQuery commands on it


yeah, i understand that parsing/reading the file can take some time


but for some reason, afterwards I can’t really work with the XML as it brings my REPL to a halt


maybe i’m doing something wrong


 (cheshire/generate-string {:bar (java.util.Date.)}))
this points and the json spec confirms that there's no such thing as date json right?


@dpsutton as far as i know, it’s a string


what is the idiomatic way to ‘walk’/‘process’ an XML file with Clojure?


should i try to think about it like a map like dataset


like the library


where you go through the dataset


where you do like (zip/down), (zip/right) etc


Data.xml is built on top of Java Stax library


Zippers are nice, but need to load the whole thing in memory


So 1mb of XML probably becomes 10mb once loaded in memory for zipper


Data.xml doesn't load it all in memory, by using Stax under the hood


But how you query the XML will matter


i was googling for an XQuery library earlier today to work with in Clojure ha


If you do a big search, even though it's lazy, it'll still need to load everything it searches over


So it might be related to how you're querying over it, for the slowness I mean


Also, at the REPL, trying to print anything too big will make everything super slow, your terminal/editor cannot handle large amount of text


latest thing i am considering is creating a server, for example with BaseX, which takes the xml file as a database, and then just query an API with clojure for the right data


yeah, i noticed that ha!


unfortunately even running (first …) on the xml file after the 1st tag


halts the REPL


How big is that tag?


The thing is, the Repl will keep trying to print things, and printing lazy things realizes them


i don’t know the size


of the tag


So if (first tag) returns a LazySeq for example, or something lazy, and you print that, it'll recursively realize it all and print everything



(def ftag (first xml))
(type ftag)


I've never used data.xml, but I think from the doc, you would call parse and then use either tree-seq or postwalk over it to extract what you want


am trying this now


(type (first xml-raw))


reading the file this way: (def xml-raw (-> filename io/file xml/parse))


i tried the example from, with io/resource, however that returned nil so didn’t work


(count xml-raw)


(first xml-raw)
[:tag :datafile]


(second xml-raw)
 {:xsi:noNamespaceSchemaLocation "wiitdb.xsd",
  :xmlns:xsi "",
  :xmlns:xs ""}]


opening the 3rd brings the REPL to a halt ha


so i can’t show that one


What is the operations you're looking to do? Just like explore the document from the Repl?


at first i was trying to do that yes, but quickly found out that wouldn’t work


now i just want to convert the XQuery I created into a Clojure ‘function’ or somthing


I'd say if that's the case, for big XML files, tens to hundreds and above megabytes, probably using Berkeley DB XML would be best.


with learning clojure, i’m taking every opportunity to use it ha


so i found this ~30 mb xml file with Wii games data and figured, let’s create an app ha


For a single query over the data, you should be able to use data.xml. But the challenge is writing an efficient query for it that only pulls in the minimum needed


so here is the working XQuery:

let $search-for := 'mario'

for $game in /datafile/game
where contains(lower-case($game/@name), $search-for)
where $game/type = ''
order by $game/@name

return <a href=''>{data($game/@name)}</a>


And at the REPL, you will constantly be at risk of accidentally printing large amount of the xml


yeah, thankfully in Calva i can interrupt running evaluation, saved me several times ha


I think you should be able to recreate that query with that


that looks very interesting, i will try right away


i get the feeling this library is out of date,


i know that Alex was updating all the Clojure libraries to 1.0.0 that were working some time ago


this one is on 0.0.8, it might not be stable yet


@patrick.glind use the 0.2.0 alpha version


If you're used to versioning in other tech, Clojure's approach can look very strange


i am using deps.edn, can’t seem to be able to find the library on Clojars


ok i will use that


yeah i read about that some time ago, also that in Clojure land, last commit > 1 year ago doesn’t mean out of date haha


Contrib libraries are not on Clojars. They're on Maven.


org.clojure/data.xml {:mvn/version "0.2.0-alpha6"}
That's in our production code at work.


why is their so many options for everything in clojure


(we switched from lein to boot in 2015 and then to CLI/`deps.edn` in 2018)


it's really not helpful for beginners.


I feel a curated list of libraries for common tasks is needed.


yeah i was going to say the same thing haha


my ‘workflow’ is usually to go to clojars to find the actual deps.edn ‘line’ to use


I have a bunch fo things I want to build, and I keep trying to build them in Clojure, but invariably hit a problem where i have no idea what library to use or some tooling just fails and somethjing that I could do in an hour or so ends up with me losing hours and hours or days trying to get it working in Clojure and I just give up


and I go do it in C#


@qmstuart Clojure is not easy. It's simple, but simple is not the same as easy.


Clojure is hard to learn and hard to master. It isn't for everyone.


I think it's mostly hard to master because the quality of documentation is quite poor though


There was a lot less documentation when many of us learned it in the past. It was even harder to learn then.


There are so many books, tutorials, and videos nowadays by comparison.


Yes, there is a lot of documentation, but a lot of it is quite shallow I find


It's OK for something to be hard to learn/master. A violin is hard, for example (a common comparison with Clojure as a "sharp tool").


I haven’t had any issues with documentation quality. I think one of the hurdles is that the documentation might be for java code that a clj library is wrapping (or js code that cljs is wrapping)


so it’s not always clear where to look, but I think I’ve gotten so used to it, that’s not really a pain point for me personally


There is certainly an underlying assumption behind a lot of the material that the developers have a fair bit of programming background. But it's slowly improving over time.


being familiar with both java and js can provide a huge boost for using clojure/clojurescript


Anyways, in case it was missed, I posted a link to which is a (lightly) curated list of libraries organized by category, in case that's helpful to either of you @patrick.glind or @qmstuart


@patrick.glind This tutorial seems pretty good. And it shows how to parse and query a 44GB XML file in Clojure:


awesome, thanks!


i will look at that tomorrow


i was able to get part of the way working


although the REPL crashed on xml-seq while following


(ns wiitdb
   [ :as io]
   [ :as data-xml]))

(def filename "wiitdb.xml")

(def xml-raw (data-xml/parse (io/reader filename))) ; this works..

(xml-seq xml-raw) ; crashes repl


this is what i have now


I'd expect that to crash the REPL: you're asking it to print (and therefore realize) the entire lazy sequence.


Yup, you basically can never let the Repl print the XML or any big parts of it or it'll choke


You could probably print (first (xml-seq xml-raw)) or maybe even (take 5 (xml-seq xml-raw))


The tutorial I linked has a to-short-xml convenient util that will print the node without it's children which you can use


@seancorfield yeah that works, doing take 5


@didibus, it’s on my todo list for tomorrow’s reading 🙂


thank you guys so much for your help!


it’s bedtime here in Netherlands haha