Fork me on GitHub
#beginners
<
2021-03-24
>
andy.fingerhut02:03:16

in-ns is reasonable for switching between namespaces that you have previously require'd

cschep03:03:20

DEPRECATED: Libs must be qualified, change compojure => compojure/compojure (deps.edn)

cschep03:03:32

hey there, just added compojure to my deps.edn

cschep03:03:42

confused what this means? thought it seems to work

cschep03:03:20

like i know i can change it to compojure/compojure but how would i have known that?

andy.fingerhut03:03:29

The deprecated message you quoted is the way I found out about it 🙂

cschep03:03:36

ha, ok awesome

cschep03:03:05

is it just double the name?

andy.fingerhut03:03:10

Leiningen and Clojure CLI tools have been permissive in allowing just an artifact name, without a group name, for many years.

andy.fingerhut03:03:45

e.g. Clojure's full name is org.clojure/clojure, where I believe org.clojure is the group name, and clojure is the artifact name within the group org.clojure.

andy.fingerhut03:03:55

Thisi is a Maven naming thing.

andy.fingerhut03:03:22

The Clojure CLI tools have recently deprecated the earlier common practice of leaving out the group name.

andy.fingerhut03:03:03

This earlier common practice was that if you didn't want to come up with a group name, then if your project's name wasn't already a group name on http://Clojars.org, then your project name also became the group name of that project.

andy.fingerhut03:03:18

(Caveat: This is my understanding, which could be off in several important details)

cschep03:03:25

that’s cool thanks

cschep03:03:38

it looks like if you go search clojars they will tell you the right string to put in

cschep03:03:42

which is nice

cschep03:03:59

so i’ve been hacking away at a single source file, core.clj which has a main function I don’t use, i’ve just been evaluating stuff in the REPL, i want to move some of this code to a differnet file and use this core to start a web server and then call into that other code

cschep03:03:08

do i make a new file that has its own namespace?

cschep03:03:29

is there a good thing i should read before i ask every individual question about this process?

andy.fingerhut03:03:06

I'm not sure of a good resource that covers this question, but would not be surprised if there was one.

andy.fingerhut03:03:54

You are on the well supported and easier to understand path if you put every Clojure namespace in a separate file, each beginning with an ns form with the name of that namespace, and the namespace name corresponds one-to-one with the file name it is stored in.

andy.fingerhut03:03:55

Whenever you do require or use in Clojure, it searches all directories in your class path for one of a few file names that correspond with the namespace name, e.g. require of a namespace foo.bar.baz , and if your project's classpath contains the directory src (a common choice, but not mandated by Clojure), then it will look for a file named src/foo/bar/baz.clj

andy.fingerhut03:03:37

I would actually recommend against using a namespace ending with core, but it is a fairly minor reason -- if you ever get a stack trace because of some exception thrown in your code, it contains not the full path names of source files, but only the last part, e.g. bar.clj, not src/foo/bar/baz.clj in the stack trace lines, so if you have a namespace foo.core, any lines in a stack trace for that file will show up as core.clj, which is the same file name for many functions built into Clojure that likely also show up in your stack trace.

cschep03:03:46

ah this is all very helpful, thank you!

cschep03:03:57

is this java stuff or does clojure do its own file loading ?

andy.fingerhut03:03:37

Warning: you are certainly allowed to have dashes - in elements of your namespace names in Clojure. You must replace those dashes with underscores when you create the corresponding file or directory names in your source code, because that is what require and use look for.

cschep03:03:16

ha, i bet that has cost some people some time

andy.fingerhut03:03:45

require is Clojure-specific, and the turning of dashes into underscores is Clojure-specific, because Clojure explicitly allows dashes in names, but Java does not allow dashes in class names. Part of the bottom of Clojure's require implementation uses a Java method getResource that searches the classpath, if I recall correctly: https://docs.oracle.com/javase/7/docs/api/java/lang/ClassLoader.html#getResource(java.lang.String)

cschep03:03:45

oh awesome

cschep03:03:01

I appreciate the explanation

cschep03:03:26

I’m copying the bones of this

andy.fingerhut03:03:26

The naming of a file core.clj is common in Clojure projects, and very well might trace back to the use of core.clj in Clojure's implementation, and/or the creation of a file core.clj when Leiningen creates a template project for you in its default template.

andy.fingerhut03:03:11

I'd recommend feeling free to rename that file and namespace in any such template.

seancorfield04:03:44

Regarding the lib name thing (`compojure` => compojure/compojure) there is another change which folks need to be aware of if they plan to publish a library (to Clojars; Maven is already stricter). Clojars historically allowed "any" group ID so a lot of early libraries just picked a single name, like ring or compojure or hiccup, and that became the group ID as well as the artifact ID (these are Maven terms really). For security reasons, Clojars is implementing a policy where group IDs must be reverse domain name format -- which is already common in the Java world -- and you must be able to verify that the reverse domain name belongs to you. By default, they offer net.clojars.<username> which everyone can use (because your username is "verified" by virtue of you logging into Clojars). In addition, if you use your GitHub ID to login to Clojars, you can use com.github.<username> for your group (verified by virtue of you logging into GitHub to authenticate).

seancorfield04:03:07

There's a good thread about this on ClojureVerse if you're interested in more about it: https://clojureverse.org/t/clojars-verified-group-names/7297

seancorfield04:03:41

When I started publishing libraries, I chose seancorfield as my group ID but I am slowing migrating my libraries to com.github.seancorfield. If you use the CLI and deps.edn, and you create projects with clj-new, the latest version tries hard to create library projects that will conform to the new security policy at Clojars. If you ask clj-new to create you a new project with the name <username>/<projectname> then it will assume you mean com.github.<username> for the group ID and will set you up to publish your library that way.

cschep04:03:46

that’s really interesting

cschep04:03:03

where does clojars end and maven begin?

seancorfield04:03:53

Maven has much tighter control over what gets published and how. Clojure itself is published to Maven, along with all the "Contrib libraries" (everything under the org.clojure group). Clojars is maintained by the Clojure community (funded by Clojurists Together and various companies) and allows "everyone" to publish libraries there. Leiningen, Boot, and the Clojure CLI all know to look on Maven for a library and if it isn't there they look on Clojars for it.

cschep04:03:26

ah that’s cool

cschep04:03:40

you don’t really have to know where the thing you want is

cschep04:03:02

i’ve been using vscode to start a REPL and jack-in

cschep04:03:30

but i’d like to learn how it actually works so i’m following

cschep04:03:46

clj -X schepball.main/-main

cschep04:03:48

is this right?

dpsutton04:03:44

doesn't look like it. -X needs to call a function that takes a single map. -main is usually a function that accepts a sequence of string args. the two aren't really comptible

seancorfield04:03:04

-main is generally invoked via "main opts" which means -M and passes zero or more strings to the function (which has [& args] for arguments); executing functions via -X which pass a single hash map to the function.

NoahTheDuke04:03:20

@seancorfield to pull out a part of the earlier conversation, how do you handle branches when developing in long-running repl sessions?

seancorfield04:03:41

It's a good question! Maybe it's our working style at World Singles Networks but I don't find it's a problem. My REPL state just followed the code that I'm working on, so when I branch for a feature/bugfix, I just keep working in the REPL, until I'm done and push the branch for a PR review. If I need to work on something that builds on that, I'll cut the new branch off the previous branch rather than the trunk, and keep working. If it's orthogonal work, the changed functions in the REPL won't affect it anyway, even I start over from the trunk.

seancorfield04:03:05

Most of our branches are fairly short-lived. We merge everything to the trunk as soon as it is reviewed and approved. And we deploy to staging for business review almost every day (sometimes several times a day), and from there almost all apps can go to production completely automatically as many times a day as business want (the business team can deploy from staging to production by checking a box and clicking a button, even for database migrations).

seancorfield04:03:19

If I have to go back and forth between two branches that actually do "interfere" a "reload all" generally re-syncs the state (literally just (require 'main.ns :reload-all) -- even while the app is running).

NoahTheDuke12:03:38

That’s super interesting! Thanks for the explanation.

cschep04:03:54

ah that is interesting

cschep04:03:51

i’m finding myself wishing for a clj run that new how to run my “main” file

phronmophobic04:03:11

I wrote a small utility with a similar idea, https://github.com/phronmophobic/clirun Args are parsed as edn, but you can specify the main function. For example:

# Write to a file using clojure.core/split
$ clj -M:run clojure.core/spit '"foo.txt"' '[1 2 3]'
$ cat foo.txt
[1 2 3]

seancorfield04:03:36

clojure -M -m my.namespace isn't much work though...

cschep04:03:23

ah i didn’t know about that flag

cschep04:03:35

as usual i should read more before asking questions 🙂

cschep04:03:59

the -m flag makes sense to me I think

cschep04:03:08

but the -M i’m not understanding

cschep04:03:31

it uses clojure.main?

dpsutton04:03:41

and clojure -X core/one-of-my-tasks works just as well. can only be a single main in a file, whereas -X can start with any function taking a single map as an arg

dpsutton04:03:28

(doc clojure.main/main)
-------------------------
clojure.main/main
([& args])
  Usage: java -cp clojure.jar clojure.main [init-opt*] [main-opt] [arg*]

  With no options or args, runs an interactive Read-Eval-Print Loop

  init options:
    -i, --init path     Load a file or resource
    -e, --eval string   Evaluate expressions in string; print non-nil values
    --report target     Report uncaught exception to "file" (default), "stderr",
                        or "none", overrides System property clojure.main.report

  main options:
    -m, --main ns-name  Call the -main function from a namespace with args

dpsutton04:03:34

so with -M you're just creating arguments for clojure.main/main. And one of those arguments is -m which expects a ns-name, and it will call the -main function of that ns with any command line args

seancorfield04:03:12

-M can have other arguments: just a script name, for example:

[email protected]:~/clojure$ cat > script.clj
(println "Hello, World!")
[email protected]:~/clojure$ clojure -M script.clj
Hello, World!
or -e to evaluate a Clojure form:
[email protected]:~/clojure$ clojure -M -e '(println "Hello, Command-Line
!")'
Hello, Command-Line!
as well as -m to specify a namespace whose -main function should be invoked.

👆 3
cschep04:03:22

oh interesting

cschep04:03:30

looks like you can skip the -M but you get a warning

cschep04:03:36

WARNING: When invoking clojure.main, use -M

seancorfield04:03:49

Yes. In the future, you'll have to use -M for clojure.main.

cschep04:03:55

easing us into it

seancorfield04:03:29

Right now, -A will also run clojure.main and :main-opts (in deps.edn aliases) but that will change at some point and it will only start a REPL.

seancorfield04:03:58

(and if you use -A for main stuff, you'll get a warning that you should use -M instead 🙂 )

cschep04:03:30

so you should use -M for aliases?

seancorfield04:03:31

-X, -M, and -A all accept aliases and combine them and then a) exec a function, b) run clojure.main, c) start a REPL with those aliases.

seancorfield04:03:29

I'm just learning about Crux, so I just started a REPL with this command:

[email protected]:~/clojure$ clojure -Sdeps '{:deps {juxt/crux-core {:mvn/version "RELEASE"}}}' -M:rebel:reveal:add-libs:dev/repl
(based on aliases in my dot-clojure repo's deps.edn file and the dev.clj startup script).

cschep04:03:39

is dev is the file and repl is the function in it?

seancorfield04:03:43

That starts Rebel Readline as my interactive REPL, starts Reveal for tap>'ing data into to visualize it, the add-libs alias brings in a branch of tools.deps.alpha that lets me add new dependencies without restarting my REPL, and :dev/repl runs my dev.clj script.

seancorfield04:03:53

No, :dev/repl is just a keyword, an alias.

cschep04:03:27

cool that is really helpful

cschep04:03:55

oh so you have aliases that you can use on any project?

seancorfield04:03:11

Yes, in ~/.clojure/deps.edn

seancorfield04:03:35

(on an XDG setup it's in ~/.config/clojure/deps.edn I believe)

cschep04:03:04

that’s cool, lots to learn!

seancorfield04:03:07

My :dev/repl alias -- via the dev.clj script -- also starts a Socket REPL so I can connect VS Code to it (using Clover).

cschep04:03:25

i’ve seen you mention that a few times, what is the difference between a socket repl and nrepl?

seancorfield04:03:54

Socket REPL is built into Clojure and has no dependencies at all. So we run them in several production processes.

seancorfield04:03:32

You just specify a JVM option when starting up a Clojure program.

seancorfield04:03:52

But most editor tooling expects nREPL. Hence I use Clover which supports Socket REPL instead.

seancorfield04:03:08

But you can also connect via telnet directly to a Socket REPL 🙂

cschep05:03:44

ha, awesome

zackteo06:03:39

Hi Everyone, may I ask what is the most common way to transmit data from Clojurescript to Clojure, vice-versa, via HTTP server? From what I understand is that EDN is converted to JSON and back? And another way is things like transit ?

seancorfield06:03:16

I think both EDN and Transit are fairly common. EDN is a bit easier, Transit is generally a bit faster (esp. if you have larger amounts of data?).

zackteo06:03:57

Can I just use ring to give a "application/edn" response?

zackteo06:03:18

Do you know if there are any resources I can look at for this? 🙂

Ian Fernandez12:03:54

Why would I use something like a record without a body?

delaguardo13:03:10

You might want to extend a protocol to support structured record

Ian Fernandez12:03:12

(defrecord MyRecord [a b c])

popeye14:03:27

I was referring this file https://medium.com/@dashora.rajnish/how-to-create-apis-in-clojure-supporting-file-upload-and-data-transformation-using-ring-and-ad40fc3ca2d0 and I found request created as below format `

{...
 :params
  {"file" {:filename     "sample.csv"
           :content-type "text/csv"
           :tempfile     #object[java.io.File ...]
           :size         51}}
...}
How can I pass value to the temp file while writing testcases?

popeye14:03:59

how can i test post request while uploading file in clojure

Stuart14:03:03

Can someone tell me what I'm doing wrong here:

(defn build-map [acc [{:keys [hb-id status group]}]]
  (if (some? (acc hb-id))
    (update-in acc [hb-id :groups] conj group)
    (assoc acc hb-id {:status status :groups #{group}})))

(let [data [{:hb-id 1 :status "ERR" :group 1}
            {:hb-id 1 :status "ERR" :group 2}
            {:hb-id 2 :status "OK" :group 1}
            {:hb-id 3 :status "INFO" :group 1}
            {:hb-id 4 :status "WRN" :group 2}
            {:hb-id 5 :status "OK" :group 2}]]
  (->> (group-by :hb-id data)
       (reduce build-map {})))
I'm trying to turn data into
{1 {:status "ERR" :groups #{1 2}}
 2 {:status "OK" :groups #{1}}
 3 {:status "INFO" :groups #{1}}
 4 {:status "WRN" :groups #{2}}
 5 {:status "OK" :groups #{2}}}
Where it's a map keyed on hb-id with each group in a set of groups.

Stuart14:03:59

I think its my update-in I'm not understanding

Stuart15:03:55

But I tried this, and this works:

(let [hb    2
      group 3
      data  {1 {:status "ERR" :groups #{1 2}}
             2 {:status "OK" :groups #{1}}
             3 {:status "INFO" :groups #{1}}
             4 {:status "WRN" :groups #{2}}
             5 {:status "OK" :groups #{2}}}]
  (update-in data [hb :groups] conj group))
=> 
{1 {:status "ERR", :groups #{1 2}},
 2 {:status "OK", :groups #{1 3}},
 3 {:status "INFO", :groups #{1}},
 4 {:status "WRN", :groups #{2}},
 5 {:status "OK", :groups #{2}}}

dpsutton15:03:35

group-by returns a map of {grouping [items]}. But in your reduce you're only expecting [items].

dpsutton15:03:06

ie, calling first on your group by yields [1 [{:hb-id 1, :status "ERR", :group 1} {:hb-id 1, :status "ERR", :group 2}]] which is a different shape than expected by your reducing function

Stuart15:03:33

ah!! Yes, I see! Thanks. I'll have a rethink of how to do this, maybe group-by isn't the way to go

dpsutton15:03:11

yes i think you could more easily just reduce over your input data

Stuart15:03:16

yeah, i think so!

dpsutton15:03:31

well maybe not. you're almost there if you build a function that turns [{:hb-id 1, :status "ERR", :group 1} {:hb-id 1, :status "ERR", :group 2}] into {:status "ERR" :groups #{1 2}} i guess. although i haven't looked at the consistency of your data

Stuart15:03:40

This seems to work, I had the syntax wrong for desttructuring as well, had an uncessary [] around the {:keys}

(defn build-map [acc {:keys [hb-id status group]}]
  (if (some? (acc hb-id))
    (update-in acc [hb-id :groups] conj group)
    (assoc acc hb-id {:status status :groups #{group}})))

(let [data [{:hb-id 1 :status "ERR" :group 1}
            {:hb-id 1 :status "ERR" :group 2}
            {:hb-id 2 :status "OK" :group 1}
            {:hb-id 3 :status "INFO" :group 1}
            {:hb-id 4 :status "WRN" :group 2}
            {:hb-id 5 :status "OK" :group 2}]]
  (reduce build-map {} data))
Thanks for your help!

dpsutton15:03:09

that looks even more wrong to me

dpsutton15:03:24

ah nevermind. the group by is gone

Stuart15:03:15

why did it let me do this?

(defn foo [acc [{:keys [bar quax]}]]
  
  )

(reduce foo {} {:bar :quax})
=> nil
What is it expecting there for the destructuring being in a vector? As opposed to:
(defn foo [acc {:keys [bar quax]}]
  
  )

dpsutton15:03:04

destructuring is as recursive as the structure. (let [[[[[foo]]]] [[[[3]]]]] foo) returns 3

dpsutton15:03:32

the binding forms to some extent mimc the shape of what you are destructuring

delaguardo15:03:41

I guess the question is about why it is not barfing

andy.fingerhut15:03:06

It is a legal destructuring form, if you want to destructure a map as a first element of a vector given as a parameter

👀 3
Stuart15:03:04

ok! Yeah, i was expecting it to either not compile or throw

andy.fingerhut15:03:06

And not only a vector, but any sequential thing. Perhaps it is calling seq on the map given as a parameter? Not sure.

dpsutton15:03:09

i think what happened is that key-value pairs are associative and index, so that allowed []. and the hb-id was not destructured against the :keys thing. There seems to be some safety added in

dpsutton15:03:50

(let [{:keys [a]} 1]) throws an error about type hinting a primitive. but the same thing inside vectors (let [[{:keys [a]}] [1]] a) returns nil

JohnJ17:03:30

Is it common to convert seqs to vectors regularly? is the performance cost high? I find myself doing it frequently to be able to randomly grab elements from a seq. Limiting myself to small subset of contructs from clojure.core to be able to keep vectors as vectors doesn't feel right

grazfather17:03:40

I think random access to a seq is the weird part

grazfather17:03:25

it’s got to be at least O(n) to convert, with a bunch of frees and allocs by my guess

dpsutton17:03:43

use the datastructure that provides efficient operations that you need

JohnJ17:03:02

they get converted to seqs 😉

dpsutton17:03:14

can you give an example? we can give some options that might help out

3
JohnJ17:03:37

well, I know how to keep vectors as vectors but most functions work on seqs, so you have have a limited API to keep vectors as vectors, why kind of example do you have in mind?

JohnJ17:03:05

Isn't it kind of obvious?

dpsutton17:03:32

not particularly. can you give an example in your actual usage where you found yourself converting a seq to a vector?

dpsutton17:03:02

I suspect i know the gist of what you're talking about but want a concrete case to delve into it

hiredman17:03:31

the answer is basically "don't write programs that mix mapping and filtering with random indexing", so @dpsutton's question is at a high level "can you show us your program that mixes mapping and filtering with random indexing so we can help you transform it into one that doesn't do that"

hiredman17:03:09

ideally your operations on seqs form a kind of pipeline

hiredman17:03:55

so that if you must convert back and forth, you have that on each end of a pipeline of a lot of operations

seancorfield17:03:59

I’m curious about the problem space where you are “frequently .. grab[bing] elements from a seq”, i.e., where you need random access, in a context of processing seqs (mapping and filtering) @jjaws?

JohnJ17:03:23

@hiredman yes, that's what I'm doing right now, doing lots of data massaging using sequence functions then converting to vectors at the end of each pipeline for random access

dpsutton17:03:47

i was going to bring up transducers as well where you can control things a bit more

hiredman17:03:28

than it sort of depends on the pipeline

dpsutton17:03:44

yes. which is why i wanted a concrete example

JohnJ17:03:04

@seancorfield working with lots of CSV stuff, transforming it, then random access on fields

seancorfield17:03:53

Can you be a bit more specific? I’d expect CSV data to end up as a sequence of hash maps, and then each row can have O(1) access to specific fields by name.

JohnJ17:03:30

@dpsutton yep, transducers help with the 'no multiple passes over the data'

hiredman17:03:09

not really, transducers and a lazy seq pipeline will have the same number of passes

dpsutton17:03:15

hiredman has a gist for this purpose here.

dpsutton17:03:29

you want to be able to query this as a database essentially

hiredman17:03:33

transducers will remove some object allocation overhead for seqs, and will turn your pipeline into a very lean vector -> vector transformation using something like into

dpsutton17:03:40

i'm reading through tim baldridge's build yourself a logic engine that creates a db like this as well.

JohnJ17:03:56

@hiredman doesn't it avoid intermediate sequences?

hiredman17:03:15

but intermediate sequences are traversed all at once

hiredman17:03:40

unless you are traversing intermediate results for some reason

JohnJ17:03:28

@seancorfield don't see the difference, both are associative, maps will still be converted to seqs

hiredman17:03:02

@jjaws I suspect @seancorfield is thinking about (map f all-rows-in-csv) and you are thinking about (map f a-row-in-a-csv)

JohnJ17:03:48

oh ok, yes, was thinking of a-row-in-a-csv vectors vs maps

hiredman17:03:59

and this is kind of why a more concrete example can help

JohnJ17:03:16

I'll try come to up with something small but not so contrived

dpsutton17:03:13

if you need a query language over your csv, you could look into datascript or your own hand rolled triple store or set/index can be useful as well

Franco Gasperino20:03:05

regarding clojure.spec, it would reason that the more common use cases would be for library design as well as external input validation. am i correct with this assumption?

seancorfield20:03:13

https://corfield.org/blog/2019/09/13/using-spec/ — we’ve been very heavy users of Spec in production as well as dev/test since it first appeared.

sova-soars-the-sora20:03:08

learning via lurking... 🙂

Aldo Nievas21:03:42

Hi all ! newbie here! anyone using neovim 5.0 and clojure-lsp ? some pretty basic config in lua ? thanks in advance.

robertfw23:03:49

if you haven't already you may want to try #lsp and/or #vim