Fork me on GitHub
#beginners
<
2020-01-27
>
jwoods02:01:40

Okay I have another really noob question. I have this data structure how do I loop over it in clojure. Coming from C# with foreach and I have tried elixir and the have "Enum.*" for everything just trying to see how you do it here.

(#clojure.data.xml.Element{:tag :COUNT, :attrs {:Records "3312"}, :content ()}, #clojure.data.xml.Element{:tag :COUNT, :attrs {:Records "3312"}, :content ()}.#clojure.data.xml.Element{:tag :COUNT, :attrs {:Records "3312"}, :content ()})

jwoods03:01:42

I figured this out. I had to map over the collection and then I could deconstruct the object.

seancorfield04:01:18

@UT3PD0UGY It's a sequence of records which you can treat like a sequence of hash maps (as you've discovered).

jwoods04:01:54

@U04V70XH6 I'm trying to filter on a attribute with filter but keep getting error because of the type clojure.lang.LazySeq I'm using the clojure.data.xml lib and parsing a string with xml/parse-str. How to I make it not lazy ?

jwoods04:01:38

(defn get-data [data] (*let* [tag(:tag data)] (filter #(= tag ":DATA") (data))))

jwoods04:01:48

That is my filter function

seancorfield04:01:16

You have (data) but you want data -- you can't "call" a collection.

seancorfield04:01:36

Also, you can't access a member of a sequence: (:tag data)

seancorfield04:01:10

You want (:tag %) inside the filter function instead of tag

seancorfield04:01:26

Also, the :tag element of each record is a keyword, not a string, so looking for "DATA" will not find any matches.

seancorfield04:01:00

You probably want #(= (:tag %) :DATA) if that's the tag you are looking for.

jwoods04:01:41

Got it thanks, I also had it wrapped in () that was causing issues as well

seancorfield04:01:40

Yup, in Clojure parentheses are not for grouping, they indicate function calls.

seancorfield04:01:05

In C#, if you have obj.method(arg1,arg2), in Clojure you'll have (method obj arg1 arg2)

jwoods04:01:42

Cool thanks for the help. I really appreciate it.

seancorfield04:01:22

Happy to help. Welcome to Clojure! 🙂

seancorfield04:01:05

BTW, you can use triple backticks around code to make blocks of it easier to read

seancorfield04:01:23

rather than single backticks like this

jwoods04:01:46

got it wow

jwoods04:01:52

little slow tonight

seancorfield04:01:57

You may need to change the *When typing code with , Enter should not send the message. With this checked use ShiftEnter to send. * option in the "Advanced" settings.

8
seancorfield04:01:36

Looks like you're an hour ahead of me -- Mountain Time?

jwoods04:01:49

yeah Idaho 😄

seancorfield04:01:13

Cool. My wife lived in Boise growing up.

jwoods04:01:51

Yep that is where I'm from. Grew up here. Now I live in a small town about 40 min west of Boise

bv03:01:59

tearing my hair out over here; i have [org.ode4j/parent "0.4.0" :extension "pom"] [org.ode4j/core "0.4.0" :extension "pom"] in my project.clj, and yet cannot for the life of me (import org.ode4j.ode.OdeHelper) in my init-ed-ns

bv03:01:53

i've even gone so far as to pull in commons-lang3 and can use eg EscapeStringUtils just fine

seancorfield04:01:22

I don't think Leiningen can pull in bundles based on a POM -- I think you need the whole series of subprojects listed as dependencies explicitly.

seancorfield04:01:58

(although you might try :classifier instead of :extensions just to see if that works)

EmmanuelOga10:01:00

@benkay a lil late but... is it possible you need to (import org.ode4j.ode OdeHelper) instead of (import org.ode4j.ode.OdeHelper) ? at least that's how it seems to work with (ns ...)

noisesmith01:01:46

this prefixing syntax requires an extra collection, and will error as you have it here

noisesmith01:01:25

user=> (import  File)
Execution error (ClassNotFoundException) at .URLClassLoader/findClass (URLClassLoader.java:381).

user=> (import ( File))
.File

EmmanuelOga10:01:58

ah maybe not needed if you are importing just one thing.

grounded_sage12:01:07

How do people normally track the progress of large collection of asynchronous requests?

grounded_sage12:01:31

I’m using clj-http and mapping over a collection of 2000. This does 2000 asyncronous requests and I would like to track progress of it somehow.

grounded_sage12:01:28

I’m currently looking at manifold and thinking I need to build a queue? Of some sort. From that maybe I could ping the queue every second to track progress?

grounded_sage12:01:17

https://clojureverse.org/t/does-clojure-have-scrapy-like-framework-for-web-scraping/3957/2 What would be the tradeoffs between say clj-http + core.async + transducers (transducers I know little about at the moment) and the one below. That borkdude uses which is aleph + manifold.

djanus13:01:56

I will shortly release the new version of Skyscraper: https://github.com/nathell/skyscraper

👍 4
grounded_sage14:01:35

I’m not so much interested in web scraping but how I’m to handle the asyncronous requests I am making

Sam14:01:05

Does anyone know how to translate the fib example in this article http://jelv.is/blog/Lazy-Dynamic-Programming/ to clojure? I have tried and failed miserably. Either suggested version, or both, would be interesting to see. I found my functional code doing dynamic programming tends to end up quite ugly so I'm looking to learn some better techniques for it.

Alex Miller (Clojure team)14:01:04

there are a bunch of variants of fib in Programming Clojure in Ch 4, although the push there is really to turn it into a lazy tail-recursive sequence, not to use memoization

❤️ 4
Alex Miller (Clojure team)14:01:10

including a memoize one

Sam14:01:10

Thank you! Both seem like great resources, and I have already considered getting Programming Clojure so I guess now is the time.

Lucas Vassalli14:01:37

Doesn't memoize cache every past result? I wonder if there's a way to use some partial memoization so it doesn't overflow.

Sam14:01:41

That would definitely be useful.

Alex Miller (Clojure team)14:01:28

you can use the core.memoize (+ core.cache) libraries to get more control

Lucas Vassalli15:01:00

That's awesome. Thanks for pointing it out. I guess https://github.com/clojure/core.memoize/wiki/Fifo should allow a competent implementation of fibonacci. Though it's just too much complexity for a problem this simple

FiVo14:01:41

@sam.hedin Have you considered memoize? That way you can implement the straightforward recurrence using a recursive function. It might be a bit more overhead but often gives simpler code.

❤️ 4
Sam14:01:41

Hmm, yes. I somewhat overlooked it because I once had a teacher say that memoization isn't the big boy way to do it (it being more caching than "regular" dp), but that is maybe an irrelevant detail when it comes to actually writing code that just does its job. I should probably let go of such details and start using memoization. Thanks!

Aviv Kotek14:01:55

> #Logging_in_prod: New to clj and not sure about logging in production apps, in a java world i'd wrap most of my methods with

{
	logger.debug("starting method to embrace %s", rightIdioms)
	....doSomething()....
	logger.debug("successfully embraced %s idioms", idioms.length)
}
From what I can see in clojure, adding logs with 'start-end' fmt will force me to use more 'do' and 'complicate' my code instead of just returning values. what are some common idioms of logging in clj? ty!

Aviv Kotek15:01:38

@ghadi What would be a good-practice-clj in your opinion?

noisesmith18:01:04

this is false, because every fn / defn / let etc. has an implicit do

noisesmith18:01:11

you can still return values

Aviv Kotek06:01:00

ofcourse, but it adds clutter

noisesmith17:01:20

what I'm saying is it looks the same in clojure as it does in your original, clojure doesn't enforce lack of side effects and the proper bodies (functions, let blocks, reify method blocks etc.) all have implicit do

Aviv Kotek09:02:04

I added a Q to SO so this will maybe useful for others :: https://stackoverflow.com/questions/60019148/idiomatic-logging-in-clojure you are welcome to reply aswell

dharrigan14:01:32

I use org.clojure/tools.logging which wraps around slf4j (amongst other implementations such as log4j2)

ghadi14:01:59

@aviv logging is a side-effect, and Clojure expressions return values, so there is a tension there that you are pointing out

dharrigan14:01:07

Then I simply have (log/infof "Starting a [%s] Foobar using [%s]." param-1 param-2)

ghadi14:01:52

this is ok IMHO, and if things get cluttered, macros can hide the clutter

ghadi14:01:59

I don't find text-based logging nearly as useful as I did 5 years ago, and now I prefer things like clojure.core/tap> when doing things locally, or tracing facilities like Honeycomb/X-Ray when in prod

ghadi14:01:28

@dharrigan there's a wonderful keynote from Bradford Cross at an early Conj where he talks about unstructured logs and how it supports a nonsense $1B log parsing industry

dharrigan14:01:31

We dump our logs into Elasticsearch for pooling everything together.

ghadi14:01:17

(log/infof "Starting a [%s] Foobar using [%s]." param-1 param-2) instead of that ^ (emit {:foobar/start ... :params [...]})

ghadi14:01:45

it's an excellent presentation ^

dharrigan14:01:33

I would be interested to know more about tracing, esp http requests and the time it takes to service them - so looking at http://honeycomb.io / x-ray

dharrigan14:01:07

Perhaps for another channel? i.e., tracing etc...?

mloughlin14:01:49

I'd like to see a real world example of tap> based logging

mloughlin15:01:08

does anyone know any?

ghadi15:01:25

REBL has UI facilities for things that get tapped

Aviv Kotek15:01:37

@ghadi of course, that's why I asked the question, let's open a thread on it

Yosevu Kilonzo15:01:24

Hello 👋 How do you do declarative/functional iteration without a collection? For example, I want to iterate base on the value 5. I can only think of recursion as the only alternative to a for loop.

ghadi15:01:05

clojure.core/iterate

hindol15:01:37

range will create a sequence for you and you can iterate over it. There is dotimes if you just want some side effects without creating any temporary sequence. The benefit of range is it can be paired with most of the language constructs.

hindol15:01:29

For example, (for [x (range 10)] x).

hindol15:01:11

doseq instead of for, if you just want some side-effects.

hindol16:01:22

Sometimes, you will find clever ways to skip for/doseq/dotimes entirely, by using map/filter etc.

hindol05:01:04

repeat also creates a temporary sequence. So, if you repeat a billion times, and also hold onto the head of the sequence intentionally or otherwise, that's a lot of memory. Coming from imperative languages where the same memory location is updated again and again for iteration, this can surprise you.

Ryan Watkins16:01:41

Hey all, I'm using CIDER but for some reason I can't jump to definitions, what could I be doing wrong?

Ryan Watkins16:01:46

my project uses Lein

papachan16:01:16

which shortcut are you using to jump to definition?

papachan16:01:49

ah it seems it use M->

Ryan Watkins16:01:23

@papachan which function is that?

papachan16:01:31

sorry cider-jump-to-var

Ryan Watkins16:01:26

@papachan weird, I don't have that

papachan16:01:29

cider-find-var is M-.

papachan16:01:36

it worked for me

Ryan Watkins16:01:04

@papachan I feel like it's dependent on project

Ryan Watkins16:01:22

@papachan I do cider-jack-in-clj and it flat out doesn't work

papachan16:01:02

now i always start projects with C-c C-x j j with lein or with clj deps.edn. But yeah i tried with lein and it worked. i have cider version 20200120.711

dpsutton16:01:07

have you evaled the ns?

dpsutton16:01:17

CIDER is runtime dependent not static

Ryan Watkins16:01:28

How do I eval the ns?

Ryan Watkins16:01:57

I just assumed it would kick me into that ns

Ryan Watkins16:01:12

thanks guys 😄

Ryan Watkins16:01:05

I eval'd the buffer and el-doc and find definitions etc started working

cider 8
jakubl17:01:42

i tried it myself - when i eval the relevant buffers C-c C-k- then the M-. is working as expected, thanks

Luis17:01:05

Hi! I’m new with clojure and working with Luminus I need to restrict the access for specific routes. Following the luminus documentation I found what I need, but the template application (luminus with +auth) is a bit different. Now I want to understand where I’ll locate the configuration? I need to modify the defstate app-routes component?

(mount/defstate app-routes
  :start
  (ring/ring-handler
    (ring/router
     [(home-routes)(admin-routes)])
    (ring/routes
      (ring/create-resource-handler
        {:path "/"})
      (wrap-content-type
        (wrap-webjars (constantly nil)))
      (ring/create-default-handler
        {:not-found
         (constantly (error-page {:status 404, :title "404 - Page not found"}))
         :method-not-allowed
         (constantly (error-page {:status 405, :title "405 - Not allowed"}))
         :not-acceptable
         (constantly (error-page {:status 406, :title "406 - Not acceptable"}))}))))

(defn app []
  (middleware/wrap-base #'app-routes))

niveauverleih17:01:08

Why does the following work as a solution for grouping anagrams: (group-by sort ["veer" "lake" "item" "kale" "mite" "ever" "holy" "ver"]) => {(\e \e \r \v) ["veer" "ever"], (\a \e \k \l) ["lake" "kale"], (\e \i \m \t) ["item" "mite"], (\h \l \o \y) ["holy"], (\e \r \v) ["ver"]} How do sort and group-by interact here?

dpsutton17:01:28

user=> (group-by first ["veer" "lake" "item" "kale" "mite" "ever" "holy" "ver"])
{\v ["veer" "ver"], \l ["lake"], \i ["item"], \k ["kale"], \m ["mite"], \e ["ever"], \h ["holy"]}
group-by first collects all the strings that start with the first letter. group-by sort will group all the strings that have the same output when called with sort. In effect, its all the strings that are made of the same letters. which is why mite and item, lake and kale, etc. are grouped into the same partition

seancorfield17:01:15

@luigi.candita There's a #luminus channel if no one here has an answer

👍 4
Luis17:01:58

Thanks!!! I’ll try here!

jakubl18:01:15

what is the common way to accomplish this in clojure (copy-keys sourcemap [:id :name :email]) => {:id 1, :name "Jon", :email ""}? I have feeling something like this already exists and i just don't know how to find it. Thanks.

noisesmith18:01:57

what does copy-keys do?

jakubl18:01:16

copies the given keys out of the sourcemap

noisesmith18:01:18

my hunch is the clojure version is select-keys

noisesmith18:01:55

it isn't named "copy-keys" because we don't usually copy things in clojure, since we can assume they were immutable we just reuse them and that's safe

jakubl18:01:57

@noisesmith indeed thanks - works like a charm e.g. (select-keys {:id 2 :name "jakub" :zip 45603} [:id :name])

👍 4
bocaj22:01:09

I’m reading in a file sent by my colleague. The file is produced by python (3), has as a Byte order mark, and linux $file text.txt describes the file as Ascii. When I slurp the file I get spaces between each character. (slurp "e:/test.txt" :encoding "UTF-8") =>…M e c h a n i c Notice the spaces between each character.

noisesmith22:01:50

what makes you think the text is UTF-8?

bocaj22:01:15

I’m sure it’s ascii, according to the $file test.txt output

noisesmith22:01:46

so the spaces don't show up in a text editor?

noisesmith22:01:01

xxd might help

bocaj22:01:13

The spaces are in the repl. I’ve spit the file and get spaces as well.

bocaj22:01:45

^^spit the string

noisesmith22:01:46

I mean, opening test.txt directly outside clojure

bocaj22:01:02

Right…the spaces aren’t there in IDEA or notepad

bocaj22:01:18

The file was produced in python3 on a linux machine

hiredman22:01:18

figuring out encoding oddities based on repl output is usually not useful, because repls tend to be sort of best effort about those things, and you end up with multiple places where characters are being encoded and decoded and you have audit it all to make sure everything is in lock step

4
hiredman22:01:09

xdd or some kind of hex editor is a great idea, or even dumping the numeric byte values use read on an inputstream

bocaj22:01:53

Ok, thanks! I was hoping it wasn’t an encoding issue, but alas. Cheers!

hiredman22:01:02

it may or may not be

hiredman22:01:42

a repl for whatever reason might display some characters as spaces that notepad or idea display as zero width

noisesmith22:01:18

btw it's xxd not xdd (I make that mistake myself all the time) - it's easy to use and eliminates encoding ambiguities immediately

hiredman22:01:19

you need to look at the literal bytes to really determine the file contents to determine what the contents should be

hiredman22:01:03

you can also take the string you get from slurp and call seq and try get the byte value of one of the characters that is rendering as a space and look it up to see what character it is

bocaj22:01:33

awesome! Thanks.

andy.fingerhut22:01:29

(map int "some string value") can be useful in such situations.

noisesmith22:01:08

(map int (slurp ...)) does remove some ambiguity, but you're still getting the numeric values after the java methods turned the original encoding into UCS-16 and we can't take for granted that conversion was correct

noisesmith22:01:29

so knowing the original bytes in the file is still useful

andy.fingerhut22:01:17

Right. Byte sequence of the raw file is needed, too. The (map int ...) in Clojure is only useful as the "what came out of the sausage grinder" part of the issue.

bocaj23:01:10

This feels like a moment I should go google and read, but this is definitely a gap in my knowledge I’d like to fill. Here is the output of xxd on the first few characters. 00000000: 5045 5253 4f4e 5f49 4409 5049 4409 5553 If you can point me resources I’d like to learn more on how to understand this. (realizing this is likely a fundamental knowedge gap).

noisesmith00:01:42

the weird thing here is that normal ascii thinks those . are tabs

user=> (apply str (map char [0x50 0x45 0x52 0x53 0x4f 0x4e 0x5f 0x49 0x44 0x09 0x50 0x49 0x44 0x09 0x55 0x53]))
"PERSON_ID\tPID\tUS"

andy.fingerhut23:01:51

Assuming that is hex dump, hex 50 is the ASCII code for "P", so I don't see any byte-order marks there.

andy.fingerhut23:01:13

Not sure of a targeted resource specifically on debugging your issue. A few years ago I got interested in what Unicode was all about, and learned enough from Wikipedia articles on UTF-8 and UTF-16 and UTF-32 encodings to get the basic idea and some of the details. There is also a Wikipedia article on byte order marks that probably has more details than I ever retained in my head.

andy.fingerhut23:01:53

Given the Windows path name "e:/file.txt" you mentioned in an earlier expression, you are likely on Windows. JVMs have a default encoding they use for some file I/O operations, and that default can be different on different operating systems, and you can in some cases change it at JVM startup, I believe, via particular command line options.

andy.fingerhut23:01:53

In your case, I suspect that providing a particular encoding explicitly when you do the file I/O will be most robust across operating systems. I am a bit surprised that (slurp "e:/test.txt" :encoding "UTF-8") still gives you the spaces.

andy.fingerhut23:01:24

What does the output of (take 20 (map int (slurp "e:/test.txt" :encoding "UTF-8"))) look like for you?

andy.fingerhut23:01:09

Or if you want to see hex to more easily compare to the output of xxd, you can try (take 20 (map (fn [c] (format "%02x" (int c))) (slurp "e:/test.txt" :encoding "UTF-8"))) instead