Fork me on GitHub
#beginners
<
2019-01-04
>
seancorfield00:01:21

@shaken In the non-alpha version, I would expect the namespaces to lose their .alpha suffix and therefore path/to/something/alpha.clj will be moved to path/to/something.clj

seancorfield00:01:52

In the spec-alpha2 repo, just FYI, the namespaces are clojure.spec-alpha2, clojure.spec-alpha2.test, clojure.spec-alpha2.gen and the files have _alpha2 in them. So the _structure_ there is closer to the post-alpha version (and the -alpha2 portion of the namespaces would be deleted, and the _alpha2 portion of the filenames would be removed).

shaken00:01:59

@seancorfield For example, the main file would be “spec/src/main/clojure/clojure/spec.clj” and the subordinate items “spec/src/main/clojure/clojure/test/spec.clj” and “spec/src/main/clojure/clojure/gen/spec.clj” ?

shaken00:01:23

@seancorfield Thank you for the alpha2 reference. I will look there, as well. I appreciate the responses!

seancorfield00:01:49

Yes, but as @hiredman said, the spec/src/main/clojure part is just the root of the source files for classpath purposes. The path of the files (relative to that) are the pieces that map to namespaces.

seancorfield00:01:13

In most Clojure projects, the root of the source files will just be <repo>/src (and the tests will be in <repo>/test -- whereas in Contrib libraries, they follow a more Maven-y style of layout and use <repo>/src/test/clojure for the tests).

shaken00:01:10

Yes, @hiredman’s statements about class path made sense! I was trying to get a sense of the structure right at the end of the path, which your comments have clarified. My real conundrum is about partitioning my application into namespaces and how to fit this with namespaced keywords. I am used to applications composed of lots of small files (less than 200 lines) and the added wrinkle of namespaced keywords makes knitting together a Clojure application… interesting. Happily, I think I am getting it worked out. The namespaces and namespaced keywords of an application make up a directed acyclic graph, and like any dependency structure take a little thought to be neat and concise.

noisesmith00:01:05

keyword namespaces don't have to have any relationship to file namespaces / file paths

shaken01:01:57

@noisesmith If you use an unqualified double-colon, you get the local namespace (filename related) and the double-colon as-name-slash format uses the required name, which is derived from the required namespace (again, filename related). (I do see there is also an arbitrary form for namespace-qualifying your keywords. Perhaps they all arbitrary and the above relationships are by-convention.) Perhaps due to my inexperience with the language, it seems that keeping namespaced keyword and namespaces (file-structure) aligned will make the program easier to understand. Please help me understand if my intuition is not a good approach .

noisesmith01:01:29

sure - those reader shortcuts exist, but they aren't based on file paths, they are based on aliases

noisesmith01:01:57

but the real point is that you don't need to use those shorthands, and you don't need to restructure things just so those shortcuts are available

seancorfield01:01:40

@shaken Also, you can create aliases (and namespaces) on the fly to use with qualified keywords -- that have nothing to do with files.

seancorfield01:01:27

(alias 'm  (create-ns 'ws.domain.member))
ws.domain.member does not need to exist here.

seancorfield01:01:58

So then you can use ::m/foo and it expands to :ws.domain.member/foo -- and you do not need ws/domain/member.clj to exist.

seancorfield01:01:11

We also use qualified keywords where the "namespace" part of it does not correspond to any namespace or alias, e.g., :wsbilling/member-id

shaken01:01:17

Cool! I guess this all falls out of keywords being constructs that resolve to their hash, so you don’t need to obtain the namespaced keyword from another file in the way you would pull in an enumerated value in a more C-like language (which is, I admit, my frame of reference). Thanks again, everyone!

noisesmith01:01:18

to nitpick, keywords don't resolve to their hash - they are cached so that equality checking is cheap, but they resolve to themselves

noisesmith01:01:43

they may cache their hash value as an optimization though(?) since they are so often used as keys

mathpunk01:01:51

Is anyone up to helping me figure out zippers? I've got an XML parsing problem

mathpunk01:01:05

Specifically, I'm looking at some output from JUnit

mathpunk01:01:20

The cases I care about are:

mathpunk01:01:44

Successes: They look like <testcase classname="App landing page" name="has the necessary components" time="13.706" /> i.e., they have no content

mathpunk01:01:37

Failures: They look like

<testcase classname="App landing page" name="can fail a test" time="10.822">
   <failure type="exception" message="Failed: This fails deliberately"><![CDATA[Error: Failed: This fails deliberately...(more)...
   </failure>
</testcase>

mathpunk02:01:05

And skipped: they look pretty much like failures

<testcase classname="App landing page" name="it can skip a test" time="0.002">
   <skipped message="Temporarily disabled with xit" />
  </testcase>

mathpunk02:01:35

That's kind of it --- I would like to count the number of successful, failing, and skipped tests, and get data out of them

noisesmith02:01:13

I'm not convinced a zipper would help here - you don't need to edit it and the structure is very predictable and shallow

noisesmith02:01:37

why not xml-seq and group-by ?

mathpunk02:01:06

I do love group-by, I don't know the former --- I will read up

noisesmith02:01:27

in fact, you might just want clojure.xml/parse instead of xml-seq

noisesmith02:01:48

I do recommend zippers in general, I just don't think they are the optimal tool for this specific task

mathpunk02:01:56

you might be right..... I have been worried that the xml might be heavily nested

mathpunk02:01:04

but it might be much flatter than i expected

noisesmith02:01:25

if heavily nested comes up, xml-seq would help, if not xml/parse is simpler

noisesmith02:01:09

if you needed to make a new xml output by restructuring a complex tree, that's where a zipper would shine

noisesmith02:01:28

if you ever tried to do complex tree restructures with immutable data, and you came up with something that was general and actually worked, you were probably 80% of the way to implementing a zipper :D

mathpunk02:01:13

🤔 I have to pre-parse my data somehow.

1. Unhandled java.net.MalformedURLException
   no protocol: <?xml version="1.0" encoding="UTF-8" ?> <testsuites disabled="0" errors="0" failures="2" tests="7"...

mathpunk02:01:23

that's from chucking a raw string at xml/parse

noisesmith02:01:49

"file, input stream, or string naming a URI"

mathpunk02:01:53

i need to use.... thinks of computer word... bytes?

mathpunk02:01:07

oh it'll take the file? that'll do

noisesmith02:01:40

if you find you need to use a string, (java.io.ByteArrayInputStream (.getBytes s)) does do the trick (fixed)

mathpunk02:01:54

aaaaaa this is so much easier than what i was trying before

mathpunk02:01:01

🙏:skin-tone-2:

noisesmith02:01:11

glad I could help

seancorfield03:01:46

👋:skin-tone-2: at @me1238 -- welcome to the Slack!

seancorfield03:01:11

(I think we met at ClojureBridge some time back?)

Jenny Kwan06:01:31

I think we've met on multiple channels. Nice to see you again!

lxsli10:01:11

Is it possible to get nREPL to print the port it's listening on? I have to have a few open at once and sometimes the port gets lost in the scrollback

lispyclouds12:01:23

@U9MDWLP5Y it writes a file called .nrepl-port with the port number in the directory it was started. That would help?

lxsli12:01:50

Perfect, thank you!

Aitor11:01:11

hi there everyone!

Aitor11:01:27

firstly, is this the right place to talk about Datomic?

Aitor11:01:32

I'm not quite sure....

kodnin11:01:05

@aaguirre, welcome! There is an active #datomic channel.

Aitor11:01:04

oh, thx for the welcome @kodnin 😎😎

Aitor11:01:56

I'll tell you my story: we are working on a project were Datomic is being used. We want to analyze de data using ML alorightms (we are data analyst). The thing is we don't event know where to start from!! Could someone point me in any direction?

absolutejam14:01:58

alright, I'm back with another one

absolutejam14:01:29

and I think the answer might blow my non-functional mind a little

absolutejam14:01:49

what's the idiomatic way to perform a loop within a loop?

absolutejam14:01:02

I guess using the word loop is a bit of a misnomer

absolutejam14:01:24

I mean, I've been working on replacing my inner logic of for each with map (usually)

danm14:01:48

I think we've got one loop in our entire codebase (with a recur). Everything else is map/`reduce` etc

absolutejam14:01:49

I guess I should say what data I'm working with

absolutejam14:01:11

well I haven't used for or loop yet

absolutejam14:01:27

and I still haven't used reduce and I'm not sure if that's because I don't fully understand it or if I just haven't needed to

lispyclouds14:01:08

could you provide the input and output?

lispyclouds14:01:58

mostly nested imperative loops can be solved using a list comprehension

absolutejam14:01:11

I think I've actually got past that point of nested loops, but I'd forgotten, haha

absolutejam14:01:32

I'm not at the point where I have 2 vectors, both with the same amount of items

absolutejam14:01:45

I want to iterate over the first and also access the same element in the second

absolutejam14:01:37

Maybe I should zip them together into a vector/hash-map so I can iterate over each matching index

absolutejam14:01:54

I wrote a function to do that haha

absolutejam14:01:14

that's the hardest part - finding the name of the existing function, haha

absolutejam14:01:25

I was looking for zip coming from Python but the results weren't what I wanted

lispyclouds14:01:46

map does that too, it can take multiple lists and work like this:

(map vector '(1 2 3) '(4 5 6))
=> ([1 4] [2 5] [3 6])

danm14:01:50

I mean, map can also be given multiple collections

danm14:01:01

Damn, @rahul080327 beat me to it 😉

absolutejam14:01:26

Didn't know that

absolutejam14:01:38

I had (apply map vector columns)

absolutejam14:01:50

because it was a vector of vectors

danm14:01:04

I basically spend most of my time on clojuredocs it seems 😉

enforser14:01:46

not sure how complete it is, but on the topic of finding core clojure functions check out https://re-find.it/

absolutejam14:01:30

So, is there no way to emulate how a traditional for each would work?

enforser14:01:35

even my first test didn't find the function I was expecting, but it does give you a whole wack of functions that are pretty close! I input "abc" and made the result '(\a \b \c), hoping to have clojure.core/seq returned to me

absolutejam14:01:43

Or do I just need to get that out of my mind and manipulate the data to a start where I don't need that?

absolutejam14:01:46

As in, the scoping

absolutejam14:01:09

Generally, I've been find with map, anonymous functions or partial/comp

absolutejam14:01:41

but the whole for x in y: providing its own scope and allowing multiple statements is easy, but perhaps not idiomatic

lispyclouds14:01:44

you can use https://clojuredocs.org/clojure.core/doseq closest thing to foreach i guess

absolutejam14:01:46

Like, could I map with do

absolutejam14:01:24

but it doesn't return and I feel like it's not the 'right' way

enforser14:01:36

use for if you want it to return something

(for [x (range 5)]
  (inc x))
=> (1 2 3 4 5)

lispyclouds14:01:54

For me, the way that works is if the code inside returns something use map otherwise use doseq or run!

danm14:01:03

I'm not following what a foreach does that you can't do with the core functions anyway

SoV415:01:08

How can i update-in a :key within a specific map located in a vector of maps?

SoV415:01:14

(using swap!)

SoV415:01:46

previously solved this using map-indexed to get a indexed list of maps...

SoV415:01:54

or keep-indexed

SoV415:01:18

are there alternatives?

jaihindhreddy15:01:26

@sova (update-in [{:key 0} {:key 1} {:key 2}] [1 :key] inc) returns [{:key 0} {:key 2} {:key 2}]

jaihindhreddy15:01:19

The docstring for update-in reads

'Updates' a value in a nested associative structure
And vectors are associative structures associate numeric keys to values

SoV415:01:51

thank you. how would i figure out that [1 :key] 1 position?

jaihindhreddy15:01:48

I don't follow. Can you expand on your problem?

SoV415:01:19

(swap! tv-state update-in [second-hit :comments] concat (:id new-comment-map)) is not doing the trick. second-hit is a search on matching ID and is an integer keeping note of position. maybe the error is different

jaihindhreddy15:01:55

And is (:id new-comment-map) a sequence of comments?

SoV415:01:59

specifically it is one map

SoV415:01:09

i found my error, it is from an earlier reset! of the atom.

SoV415:01:20

it changes the atom from a vector of maps to a list(?)

SoV415:01:33

how can i conj more maps onto a vector of maps?

jaihindhreddy15:01:37

(let [more-maps [{:a 1} {:a 2}]] (concat [{:a 0} {:a 3}] more-maps)) gives you ({:a 0} {:a 3} {:a 1} {:a 2})

SoV416:01:06

Thank you

mfikes16:01:54

I’d also consider just using into for this one.

(let [more-maps [{:a 1} {:a 2}]]
  (into [{:a 0} {:a 3}] more-maps))

mfikes16:01:48

That leads to a simple (swap! my-atom into some-coll) pattern

Lucas Barbosa17:01:41

What is an idiomatic way to configure my clojure application? Is there any "standard"? I want to be able to swap config files with db addresses, usernames and passwords to change the behavior of the application

devo17:01:41

There are a number of libraries commonly used for configuration, but I personally prefer https://github.com/juxt/aero

Lucas Barbosa17:01:12

I tried environ, but it does not make it easy to simply read a config file

Lucas Barbosa17:01:22

I will try aero now

Lucas Barbosa17:01:11

Exactly what I wanted, thanks @U1RUG108P

victorb17:01:58

How can I track down where my spec error is coming from? Including a new dependency so I know it's one of it's dependencies, but spec doesn't seem to tell me which one...

Caused by: clojure.lang.ExceptionInfo: Call to clojure.core/ns did not conform to spec. {:clojure.spec.alpha/problems ({:path [:clauses :refer-clojure :clause], :pred #{:refer-clojure}, :val import, :via [:clojure.core.specs.alpha/ns-refer-clojure :clojure.core.specs.alpha/ns-refer-clojure], :in [2 0]} {:path [:clauses :require :clause], :pred #{:require}, :val import, :via [:clojure.core.specs.alpha/ns-require :clojure.core.specs.alpha/ns-require], :in [2 0]} {:path [:clauses :import :clause], :pred #{:import}, :val import, :via [:clojure.core.specs.alpha/ns-import :clojure.core.specs.alpha/ns-import], :in [2 0]} {:path [:clauses :use :clause], :pred #{:use}, :val import, :via [:clojure.core.specs.alpha/ns-use :clojure.core.specs.alpha/ns-use], :in [2 0]} {:path [:clauses :refer :clause], :pred #{:refer}, :val import, :via [:clojure.core.specs.alpha/ns-refer :clojure.core.specs.alpha/ns-refer], :in [2 0]} {:path [:clauses :load :clause], :pred #{:load}, :val import, :via [:clojure.core.specs.alpha/ns-load :clojure.core.specs.alpha/ns-load], :in [2 0]} {:path [:clauses :gen-class :clause], :pred #{:gen-class}, :val import, :via [:clojure.core.specs.alpha/ns-gen-class :clojure.core.specs.alpha/ns-gen-class], :in [2 0]}), :clojure.spec.alpha/spec #object[clojure.spec.alpha$regex_spec_impl$reify__2509 0x48b22fd4 "[email protected]"], :clojure.spec.alpha/value (clojure.tools.namespace.find (:require [ :as io] [clojure.set :as set] [clojure.tools.namespace.file :as file] [clojure.tools.namespace.parse :as parse]) (import ( File FileReader BufferedReader PushbackReader InputStreamReader) (java.util.jar JarFile JarEntry))), :clojure.spec.alpha/args (clojure.tools.namespace.find (:require [ :as io] [clojure.set :as set] [clojure.tools.namespace.file :as file] [clojure.tools.namespace.parse :as parse]) (import ( File FileReader BufferedReader PushbackReader InputStreamReader) (java.util.jar JarFile JarEntry)))}

alexmiller17:01:05

the namespace with the problem is clojure.tools.namespace.find

alexmiller17:01:12

I believe that’s something that was fixed long ago

victorb17:01:34

yeah, I'm pretty sure updating one of my dependency's dependency will fix the issue, but wanted to pinpoint which one

alexmiller17:01:55

the actual problem is the use of import instead of :import in the ns decl

victorb17:01:25

yeah, not the first time I encounter the error myself. The issue is finding where this is. Are you saying it's just related with the clojure core version?

alexmiller17:01:53

no, the error is in the org.clojure/tools.namespace dependency

victorb17:01:50

[digitalocean "1.2"]
   [cheshire "5.3.1"]
     [com.fasterxml.jackson.core/jackson-core "2.3.1"]
     [com.fasterxml.jackson.dataformat/jackson-dataformat-smile "2.3.1"]
     [tigris "0.1.1"]
   [http-kit "2.1.18"]
   [midje "1.6.0"]
     [bultitude "0.2.2"]
       [org.tcrawley/dynapath "0.2.3"]
     [colorize "0.1.1" :exclusions [[org.clojure/clojure]]]
     [gui-diff "0.5.0"]
       [org.clojars.trptcolin/sjacket "0.1.3" :exclusions [[org.clojure/clojure]]]
         [net.cgrand/parsley "0.9.1"]
         [net.cgrand/regex "1.1.0"]
     [ordered "1.2.0" :exclusions [[org.clojure/clojure]]]
     [org.clojure/core.unify "0.5.2" :exclusions [[org.clojure/clojure]]]
     [org.clojure/math.combinatorics "0.0.1"]
     [org.clojure/tools.namespace "0.2.2"]
     [slingshot "0.10.3"]
     [swiss-arrows "0.1.0"]
     [utilize "0.2.3" :exclusions [[org.clojure/clojure]]]
   [prismatic/schema "0.2.0"]
     [potemkin "0.3.2"]
This is the dependency tree for digitalocean which is the dep with the issue. So updating org.clojure/tools.namespace would fix the issue. Can I override which version digitalocean is using without having to fork/publish my own version?

alexmiller17:01:16

yep - just include the most recent version in your list of dependencies - that will override it

hiredman17:01:05

that digitalocean library looks like it is including a lot of test dependencies 😕

victorb17:01:34

@alexmiller sweet, that did the trick 🙂 Thanks a lot

victorb17:01:49

@hiredman indeed, and in the end only a small part of the lib is actually tested...

hiredman17:01:26

I would atleast exclude midje there

alexmiller17:01:53

that would also probably be sufficient (unless you’re using midje)

victorb17:01:55

ah, did not know about the exclude feature, that's very handy

alexmiller17:01:00

fwiw, the error message here should be much better on Clojure 1.10

victorb17:01:23

strange, I'm on 1.10.0 already

alexmiller17:01:07

also depends a bit on where you’re seeing (if you’re in an editor or using a build tool, etc)

victorb17:01:36

error showed when doing lein ring server-headless

victorb17:01:41

so in terminal

alexmiller18:01:42

huh, yeah I can repro that with a minimal ring server app

alexmiller18:01:51

although the very first line of the error is:

alexmiller18:01:53

Exception in thread "main" Syntax error macroexpanding clojure.core/ns at (clojure/tools/namespace/find.clj:9:1).

alexmiller18:01:21

there are some things in that output that surprise me though, so maybe I’ll turn this into a ticket or two if I can figure out the cause

alexmiller18:01:07

Logged here https://dev.clojure.org/jira/browse/CLJ-2463 - this ultimately is about clojure.main being better when using -m to launch. I feel dumb that we didn’t fix this in 1.10.

alexmiller18:01:37

perhaps the most surprising is that it is loading this namespace when no code is using it, just merely by adding it as a dep

noisesmith18:01:36

isn't the idea of the lein-ring tool to do a kind of DI to create a local webserver as a convenience for apps that would deploy to a container? If I'm remembering this correctly,, I could imagine it doing spooky startup magic

alexmiller18:01:00

oh, there’s definitely some spookiness

hiredman18:01:03

server-headless runs the server locally, and uses tools.namespace to reload changed code

alexmiller18:01:23

ah, I was wondering if it was specific to the fact that this is specifically tools.namespace

Eccentric J18:01:27

What’s the best way to handle reading a file line-by-line, then writing back to the same file line-by-line without having to store the entire contents in memory?

alexmiller18:01:16

use a RandomAccessFile?

noisesmith18:01:36

write to a second file and move it when done?

noisesmith18:01:16

my concern with RandomAccessFile is that if your unit of work is a "line" this is a bad match for a datatype of bytes in terms of random access usage

noisesmith18:01:09

eg. if you increase the length of a line, I'd fear that would overwrite the next line

Eccentric J18:01:02

The line length will change between each line.

noisesmith18:01:39

yeah, I think you could do that with RandomAccessFile but it seems text-editor-complete as a task

dpsutton18:01:50

your editor will be in overwrite mode rather than insert mode

Eccentric J18:01:56

Thanks for the help. Moving a file within clojure\java is something I’ll need to look up how to do but on the other hand, maybe storing a line-seq list in memory is the best solution for my use-case? If I’m writing to one file then moving it to replace another file then I’m doing at least 2x more IO operations?

noisesmith18:01:36

on the OS level the move is just reassigning a pointer

noisesmith18:01:16

Your doing the same number of reads and writes, just to two files and temporarily using more inodes (the inodes for the input file are freed when you move the new file)

Eccentric J18:01:17

Ah I found the renameTo method of java.io.File

noisesmith18:01:29

sounds like the ticket to me

Eccentric J18:01:36

Agreed…err convinced rather. Thanks for explaining that, OS level ops is definitely not an area I’m familiar with.

noisesmith18:01:49

caveat - this assumes a sane file system implementation, and both files being created on the same fs

Eccentric J18:01:01

Ah, yes, the use case fits that criteria.

noisesmith18:01:23

a gotcha there is that sometimes the default tmp directory is not on the same file-system as regular files, which might be OK if you don't mind the extra copying when you move the file

Eccentric J18:01:15

Good to know. I think for the use case, renaming remotes in any .git/config files found in a directory, potentially contaminating a project dir or .git folder is a worse side effect than extra copying if the tmp directory is not on the same file-system

Eccentric J19:01:30

(defn update-remotes
  "Reads a .git/config file and writes updated config to temp file."
  [{:keys [src dest] :as update}]
  (let [count (atom 0)]
    (with-open [r (io/reader src)
                w (io/writer dest)]
      (doseq [line (line-seq r)]
        (.write w (replace-remote update line))
        (swap! count inc)))
    (assoc update :count @count)))
Is there a better way to count lines in that function?

noisesmith19:01:49

perhaps (reduce (fn [n line] (.write w (replace-remote update line)) (inc n)) 0 (line-seq r)) (fixed)

Prakash19:01:32

@jayzawrotny not sure what your use case is, but you can also use jgit lib to read/write git config from java code

Eccentric J19:01:39

That’s a great recommendation but it may be a bit much for my use case, plus an excellent learning opportunity.

haywood21:01:26

anyone know how to translate this Java into Clojure? new Compute.Builder

haywood21:01:42

the class is Compute.Builder

noisesmith22:01:05

(Compute$Builder.)

noisesmith22:01:27

inner classes are really just classes with $ in the name on the bytecode level

noisesmith22:01:04

if you need to import, Compute$Builder is the class name, importing Compute doesn't bring it into scope, amazingly enough

haywood22:01:20

oh then woops

noisesmith22:01:08

@haywood what's going on is literally that the java compiler generates a new class called Foo$Bar if Foo defines a class called Bar inside it, with all that implies

haywood22:01:34

hmm, that seems like good info for the official interop page

noisesmith22:01:17

there's a lot of weird gotchas with the JVM, it's hard to know where the part to document for Clojure ends and the JVM trivia starts sometimes

haywood22:01:59

yea good point

haywood22:01:16

well I really appreciate the explanation, makes sense now

Lennart Buit22:01:36

inner-classes are a bit of a hack in JVM bytecode :’)

noisesmith22:01:11

almost as good as generics (those are easiest to deal with - they simply don't exist in any way that matters for clojure)

noisesmith22:01:48

until you run into a reflective lib that looks for the metadata associated with a generic to guide its behavior...

Lennart Buit22:01:29

wasn’t it something like that a inner class can capture private variables from its outer class, and that generates all sorts of accessors in the outer class? I think I went down the “oh how are inner classes implemented in java bytecode”-rabbithole before

noisesmith22:01:16

yeah - there's interesting stuff with perms going on right? (reading the link)

Lennart Buit22:01:52

yeah there is a public accessor with a name that is not a valid java ID

Lennart Buit22:01:52

which … interestingly … is a static method instead of an instance method

noisesmith22:01:57

I'm sure it makes the resulting bytecode simpler or something

Lennart Buit22:01:21

(In my studies I had to write a compiler from a toy language to java byte code, was great fun!)