Clojurians Log v2

Why would I use something like a record without a body?

(defrecord MyRecord [a b c])

I was referring this file https://medium.com/@dashora.rajnish/how-to-create-apis-in-clojure-supporting-file-upload-and-data-transformation-using-ring-and-ad40fc3ca2d0 and I found request created as below format `

{...
 :params
  {"file" {:filename     "sample.csv"
           :content-type "text/csv"
           :tempfile     #object[java.io.File ...]
           :size         51}}
...}

how can i test post request while uploading file in clojure

Can someone tell me what I'm doing wrong here:

(defn build-map [acc [{:keys [hb-id status group]}]]
  (if (some? (acc hb-id))
    (update-in acc [hb-id :groups] conj group)
    (assoc acc hb-id {:status status :groups #{group}})))

(let [data [{:hb-id 1 :status "ERR" :group 1}
            {:hb-id 1 :status "ERR" :group 2}
            {:hb-id 2 :status "OK" :group 1}
            {:hb-id 3 :status "INFO" :group 1}
            {:hb-id 4 :status "WRN" :group 2}
            {:hb-id 5 :status "OK" :group 2}]]
  (->> (group-by :hb-id data)
       (reduce build-map {})))

{1 {:status "ERR" :groups #{1 2}}
 2 {:status "OK" :groups #{1}}
 3 {:status "INFO" :groups #{1}}
 4 {:status "WRN" :groups #{2}}
 5 {:status "OK" :groups #{2}}}

I think its my update-in I'm not understanding

But I tried this, and this works:

(let [hb    2
      group 3
      data  {1 {:status "ERR" :groups #{1 2}}
             2 {:status "OK" :groups #{1}}
             3 {:status "INFO" :groups #{1}}
             4 {:status "WRN" :groups #{2}}
             5 {:status "OK" :groups #{2}}}]
  (update-in data [hb :groups] conj group))
=> 
{1 {:status "ERR", :groups #{1 2}},
 2 {:status "OK", :groups #{1 3}},
 3 {:status "INFO", :groups #{1}},
 4 {:status "WRN", :groups #{2}},
 5 {:status "OK", :groups #{2}}}

group-by returns a map of {grouping [items]}. But in your reduce you're only expecting [items].

ie, calling first on your group by yields [1 [{:hb-id 1, :status "ERR", :group 1} {:hb-id 1, :status "ERR", :group 2}]] which is a different shape than expected by your reducing function

ah!! Yes, I see! Thanks. I'll have a rethink of how to do this, maybe group-by isn't the way to go

yes i think you could more easily just reduce over your input data

yeah, i think so!

well maybe not. you're almost there if you build a function that turns [{:hb-id 1, :status "ERR", :group 1} {:hb-id 1, :status "ERR", :group 2}] into {:status "ERR" :groups #{1 2}} i guess. although i haven't looked at the consistency of your data

This seems to work, I had the syntax wrong for desttructuring as well, had an uncessary [] around the {:keys}

(defn build-map [acc {:keys [hb-id status group]}]
  (if (some? (acc hb-id))
    (update-in acc [hb-id :groups] conj group)
    (assoc acc hb-id {:status status :groups #{group}})))

(let [data [{:hb-id 1 :status "ERR" :group 1}
            {:hb-id 1 :status "ERR" :group 2}
            {:hb-id 2 :status "OK" :group 1}
            {:hb-id 3 :status "INFO" :group 1}
            {:hb-id 4 :status "WRN" :group 2}
            {:hb-id 5 :status "OK" :group 2}]]
  (reduce build-map {} data))

that looks even more wrong to me

ah nevermind. the group by is gone

why did it let me do this?

(defn foo [acc [{:keys [bar quax]}]]
  
  )

(reduce foo {} {:bar :quax})
=> nil

(defn foo [acc {:keys [bar quax]}]
  
  )

destructuring is as recursive as the structure. (let [[[[[foo]]]] [[[[3]]]]] foo) returns 3

the binding forms to some extent mimc the shape of what you are destructuring

I guess the question is about why it is not barfing

It is a legal destructuring form, if you want to destructure a map as a first element of a vector given as a parameter

ok! Yeah, i was expecting it to either not compile or throw

And not only a vector, but any sequential thing. Perhaps it is calling seq on the map given as a parameter? Not sure.

i think what happened is that key-value pairs are associative and index, so that allowed []. and the hb-id was not destructured against the :keys thing. There seems to be some safety added in

(let [{:keys [a]} 1]) throws an error about type hinting a primitive. but the same thing inside vectors (let [[{:keys [a]}] [1]] a) returns nil

Is it common to convert seqs to vectors regularly? is the performance cost high? I find myself doing it frequently to be able to randomly grab elements from a seq. Limiting myself to small subset of contructs from clojure.core to be able to keep vectors as vectors doesn't feel right

I think random access to a seq is the weird part

it’s got to be at least O(n) to convert, with a bunch of frees and allocs by my guess

use the datastructure that provides efficient operations that you need

they get converted to seqs 😉

can you give an example? we can give some options that might help out

well, I know how to keep vectors as vectors but most functions work on seqs, so you have have a limited API to keep vectors as vectors, why kind of example do you have in mind?

Isn't it kind of obvious?

not particularly. can you give an example in your actual usage where you found yourself converting a seq to a vector?

I suspect i know the gist of what you're talking about but want a concrete case to delve into it

the answer is basically "don't write programs that mix mapping and filtering with random indexing", so @dpsutton's question is at a high level "can you show us your program that mixes mapping and filtering with random indexing so we can help you transform it into one that doesn't do that"

ideally your operations on seqs form a kind of pipeline

so that if you must convert back and forth, you have that on each end of a pipeline of a lot of operations

I’m curious about the problem space where you are “frequently .. grab[bing] elements from a seq”, i.e., where you need random access, in a context of processing seqs (mapping and filtering) @jjaws?

@hiredman yes, that's what I'm doing right now, doing lots of data massaging using sequence functions then converting to vectors at the end of each pipeline for random access

i was going to bring up transducers as well where you can control things a bit more

than it sort of depends on the pipeline

yes. which is why i wanted a concrete example

@seancorfield working with lots of CSV stuff, transforming it, then random access on fields

Can you be a bit more specific? I’d expect CSV data to end up as a sequence of hash maps, and then each row can have O(1) access to specific fields by name.

@dpsutton yep, transducers help with the 'no multiple passes over the data'

not really, transducers and a lazy seq pipeline will have the same number of passes

hiredman has a gist for this purpose here.

you want to be able to query this as a database essentially

transducers will remove some object allocation overhead for seqs, and will turn your pipeline into a very lean vector -> vector transformation using something like into

i'm reading through tim baldridge's build yourself a logic engine that creates a db like this as well.

@hiredman doesn't it avoid intermediate sequences?

it does

but intermediate sequences are traversed all at once

unless you are traversing intermediate results for some reason

@seancorfield don't see the difference, both are associative, maps will still be converted to seqs

@jjaws I suspect @seancorfield is thinking about (map f all-rows-in-csv) and you are thinking about (map f a-row-in-a-csv)

oh ok, yes, was thinking of a-row-in-a-csv vectors vs maps

and this is kind of why a more concrete example can help

I'll try come to up with something small but not so contrived

if you need a query language over your csv, you could look into datascript or your own hand rolled triple store or set/index can be useful as well

regarding clojure.spec, it would reason that the more common use cases would be for library design as well as external input validation. am i correct with this assumption?

https://corfield.org/blog/2019/09/13/using-spec/ — we’ve been very heavy users of Spec in production as well as dev/test since it first appeared.

learning via lurking... 🙂

Hi all ! newbie here! anyone using neovim 5.0 and clojure-lsp ? some pretty basic config in lua ? thanks in advance.

welcome 🙂

in-ns is reasonable for switching between namespaces that you have previously require'd

DEPRECATED: Libs must be qualified, change compojure => compojure/compojure (deps.edn)

hey there, just added compojure to my deps.edn

confused what this means? thought it seems to work

like i know i can change it to compojure/compojure but how would i have known that?

The deprecated message you quoted is the way I found out about it 🙂

ha, ok awesome

is it just double the name?

Leiningen and Clojure CLI tools have been permissive in allowing just an artifact name, without a group name, for many years.

e.g. Clojure's full name is org.clojure/clojure, where I believe org.clojure is the group name, and clojure is the artifact name within the group org.clojure.

Thisi is a Maven naming thing.

ahh

The Clojure CLI tools have recently deprecated the earlier common practice of leaving out the group name.

This earlier common practice was that if you didn't want to come up with a group name, then if your project's name wasn't already a group name on http://Clojars.org, then your project name also became the group name of that project.

(Caveat: This is my understanding, which could be off in several important details)

that’s cool thanks

it looks like if you go search clojars they will tell you the right string to put in

deps.edn

which is nice

🙂

so i’ve been hacking away at a single source file, core.clj which has a main function I don’t use, i’ve just been evaluating stuff in the REPL, i want to move some of this code to a differnet file and use this core to start a web server and then call into that other code

do i make a new file that has its own namespace?

is there a good thing i should read before i ask every individual question about this process?

I'm not sure of a good resource that covers this question, but would not be surprised if there was one.

You are on the well supported and easier to understand path if you put every Clojure namespace in a separate file, each beginning with an ns form with the name of that namespace, and the namespace name corresponds one-to-one with the file name it is stored in.

Whenever you do require or use in Clojure, it searches all directories in your class path for one of a few file names that correspond with the namespace name, e.g. require of a namespace foo.bar.baz , and if your project's classpath contains the directory src (a common choice, but not mandated by Clojure), then it will look for a file named src/foo/bar/baz.clj

I would actually recommend against using a namespace ending with core, but it is a fairly minor reason -- if you ever get a stack trace because of some exception thrown in your code, it contains not the full path names of source files, but only the last part, e.g. bar.clj, not src/foo/bar/baz.clj in the stack trace lines, so if you have a namespace foo.core, any lines in a stack trace for that file will show up as core.clj, which is the same file name for many functions built into Clojure that likely also show up in your stack trace.

ah this is all very helpful, thank you!

is this java stuff or does clojure do its own file loading ?

Warning: you are certainly allowed to have dashes - in elements of your namespace names in Clojure. You must replace those dashes with underscores when you create the corresponding file or directory names in your source code, because that is what require and use look for.

ha, i bet that has cost some people some time

require is Clojure-specific, and the turning of dashes into underscores is Clojure-specific, because Clojure explicitly allows dashes in names, but Java does not allow dashes in class names. Part of the bottom of Clojure's require implementation uses a Java method getResource that searches the classpath, if I recall correctly: https://docs.oracle.com/javase/7/docs/api/java/lang/ClassLoader.html#getResource(java.lang.String)

oh awesome

I appreciate the explanation

https://github.com/abedra/shouter

I’m copying the bones of this

The naming of a file core.clj is common in Clojure projects, and very well might trace back to the use of core.clj in Clojure's implementation, and/or the creation of a file core.clj when Leiningen creates a template project for you in its default template.

I'd recommend feeling free to rename that file and namespace in any such template.

Regarding the lib name thing (`compojure` => compojure/compojure) there is another change which folks need to be aware of if they plan to publish a library (to Clojars; Maven is already stricter). Clojars historically allowed "any" group ID so a lot of early libraries just picked a single name, like ring or compojure or hiccup, and that became the group ID as well as the artifact ID (these are Maven terms really). For security reasons, Clojars is implementing a policy where group IDs must be reverse domain name format -- which is already common in the Java world -- and you must be able to verify that the reverse domain name belongs to you. By default, they offer net.clojars.<username> which everyone can use (because your username is "verified" by virtue of you logging into Clojars). In addition, if you use your GitHub ID to login to Clojars, you can use com.github.<username> for your group (verified by virtue of you logging into GitHub to authenticate).

There's a good thread about this on ClojureVerse if you're interested in more about it: https://clojureverse.org/t/clojars-verified-group-names/7297

When I started publishing libraries, I chose seancorfield as my group ID but I am slowing migrating my libraries to com.github.seancorfield. If you use the CLI and deps.edn, and you create projects with clj-new, the latest version tries hard to create library projects that will conform to the new security policy at Clojars. If you ask clj-new to create you a new project with the name <username>/<projectname> then it will assume you mean com.github.<username> for the group ID and will set you up to publish your library that way.

that’s really interesting

where does clojars end and maven begin?

Maven has much tighter control over what gets published and how. Clojure itself is published to Maven, along with all the "Contrib libraries" (everything under the org.clojure group). Clojars is maintained by the Clojure community (funded by Clojurists Together and various companies) and allows "everyone" to publish libraries there. Leiningen, Boot, and the Clojure CLI all know to look on Maven for a library and if it isn't there they look on Clojars for it.

(by default)

ah that’s cool

you don’t really have to know where the thing you want is

but i’d like to learn how it actually works so i’m following

https://www.clojure.org/guides/deps_and_cli

clj -X schepball.main/-main

is this right?

doesn't look like it. -X needs to call a function that takes a single map. -main is usually a function that accepts a sequence of string args. the two aren't really comptible

ahh

-main is generally invoked via "main opts" which means -M and passes zero or more strings to the function (which has [& args] for arguments); executing functions via -X which pass a single hash map to the function.

@seancorfield to pull out a part of the earlier conversation, how do you handle branches when developing in long-running repl sessions?

ah that is interesting

i’m finding myself wishing for a clj run that new how to run my “main” file

Hi Everyone, may I ask what is the most common way to transmit data from Clojurescript to Clojure, vice-versa, via HTTP server? From what I understand is that EDN is converted to JSON and back? And another way is things like transit ?