Fork me on GitHub
#beginners
<
2020-10-06
>
suren01:10:09

$ lein ring server-headless
2020-10-06 12:36:28.980:INFO::main: Logging initialized @1474ms
Exception in thread "main" Syntax error compiling at (/private/var/folders/9d/3zkfrpnj15j6j2qnr53jqsmw0000gn/T/form-init2589101478955306490.clj:1:125).
	at clojure.lang.Compiler.load(Compiler.java:7647)
	at clojure.lang.Compiler.loadFile(Compiler.java:7573)
	at clojure.main$load_script.invokeStatic(main.clj:452)
	at clojure.main$init_opt.invokeStatic(main.clj:454)
	at clojure.main$init_opt.invoke(main.clj:454)
	at clojure.main$initialize.invokeStatic(main.clj:485)
	at clojure.main$null_opt.invokeStatic(main.clj:519)
	at clojure.main$null_opt.invoke(main.clj:516)
	at clojure.main$main.invokeStatic(main.clj:598)
	at clojure.main$main.doInvoke(main.clj:561)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.lang.Var.applyTo(Var.java:705)
	at clojure.main.main(main.java:37)
Caused by: java.lang.AssertionError: Assert failed: self-referential dependency
(not (= x dep))
	at ns_tracker.dependency$depend.invokeStatic(dependency.clj:54)
	at ns_tracker.dependency$depend.invoke(dependency.clj:48)
	at ns_tracker.dependency$depend$fn__1319.invoke(dependency.clj:59)
	at clojure.core.protocols$naive_seq_reduce.invokeStatic(protocols.clj:62)
	at clojure.core.protocols$interface_or_naive_reduce.invokeStatic(protocols.clj:72)
	at clojure.core.protocols$fn__8144.invokeStatic(protocols.clj:169)
	at clojure.core.protocols$fn__8144.invoke(protocols.clj:124)
	at clojure.core.protocols$fn__8099$G__8094__8108.invoke(protocols.clj:19)
	at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31)
	at clojure.core.protocols$fn__8129.invokeStatic(protocols.clj:75)
	at clojure.core.protocols$fn__8129.invoke(protocols.clj:75)
	at clojure.core.protocols$fn__8073$G__8068__8086.invoke(protocols.clj:13)
	at clojure.core$reduce.invokeStatic(core.clj:6828)
	at clojure.core$reduce.invoke(core.clj:6810)
	at ns_tracker.dependency$depend.invokeStatic(dependency.clj:59)
	at ns_tracker.dependency$depend.doInvoke(dependency.clj:48)
	at clojure.lang.RestFn.applyTo(RestFn.java:146)
	at clojure.core$apply.invokeStatic(core.clj:669)
	at clojure.core$apply.invoke(core.clj:660)
	at ns_tracker.core$add_to_dep_graph$fn__1784.invoke(core.clj:53)
	at clojure.lang.PersistentVector.reduce(PersistentVector.java:343)
	at clojure.core$reduce.invokeStatic(core.clj:6827)
	at clojure.core$reduce.invoke(core.clj:6810)
	at ns_tracker.core$add_to_dep_graph.invokeStatic(core.clj:50)
	at ns_tracker.core$add_to_dep_graph.invoke(core.clj:49)
	at ns_tracker.core$update_dependency_graph.invokeStatic(core.clj:62)
	at ns_tracker.core$update_dependency_graph.invoke(core.clj:59)
	at ns_tracker.core$ns_tracker.invokeStatic(core.clj:90)
	at ns_tracker.core$ns_tracker.invoke(core.clj:79)
	at ns_tracker.core$ns_tracker.invokeStatic(core.clj:84)
	at ns_tracker.core$ns_tracker.invoke(core.clj:79)
	at ring.middleware.reload$reloader.invokeStatic(reload.clj:8)
	at ring.middleware.reload$reloader.invoke(reload.clj:7)
	at ring.middleware.reload$wrap_reload.invokeStatic(reload.clj:34)
	at ring.middleware.reload$wrap_reload.invoke(reload.clj:21)
	at ring.server.standalone$add_auto_reload.invokeStatic(standalone.clj:61)
	at ring.server.standalone$add_auto_reload.invoke(standalone.clj:59)
	at ring.server.standalone$add_middleware.invokeStatic(standalone.clj:72)
	at ring.server.standalone$add_middleware.invoke(standalone.clj:69)
	at ring.server.standalone$serve.invokeStatic(standalone.clj:95)
	at ring.server.standalone$serve.doInvoke(standalone.clj:75)
	at clojure.lang.RestFn.invoke(RestFn.java:423)
	at ring.server.leiningen$serve.invokeStatic(leiningen.clj:19)
	at ring.server.leiningen$serve.invoke(leiningen.clj:16)
	at user$eval4205.invokeStatic(form-init2589101478955306490.clj:1)
	at user$eval4205.invoke(form-init2589101478955306490.clj:1)
	at clojure.lang.Compiler.eval(Compiler.java:7176)
	at clojure.lang.Compiler.eval(Compiler.java:7166)
	at clojure.lang.Compiler.load(Compiler.java:7635)
	... 12 more
Subprocess failed (exit code: 1)

suren01:10:09

I am clueless as this stack trace points no where in my code. Not sure whats wrong. Any help is appreciated. Thanks

suren02:10:36

Answering my own question in the hope it will be helpful to others. Turns out two of my clojure files were having save namespace. Apparently ring checks all the clojure files to make sure everything is alright.

seancorfield04:10:43

For future reference, you can see that in the stacktrace:

...
Caused by: java.lang.AssertionError: Assert failed: self-referential dependency
(not (= x dep))
	at ns_tracker.dependency$depend.invokeStatic(dependency.clj:54)
...
	at ns_tracker.core$ns_tracker.invoke(core.clj:79)
	at ring.middleware.reload$reloader.invokeStatic(reload.clj:8)
So you can see the Ring reload middleware function is calling into the ns-tracker code and that's what raised the "self-referential dependency"

seancorfield04:10:43

My recommendation would be: 1) do not use the ring plugin for lein -- learn how to start Jetty from your -main and use lein run to start it 2) do not use reloading middleware -- learn to develop a "REPL-driven" workflow so you don't need any of these auto-reload/refresh plugins etc Both of these will make you a better Clojure developer and you'll be relying less on "magic" -- when those "easy" things go wrong, you won't know how to fix them; if you can do this without the "magic" pieces, you'll have more control and more insight into the process.

seancorfield04:10:55

In particular, if you had a tight REPL-driven workflow, you would probably have caught the namespace problem immediately, in your editor, via the REPL -- before even trying to run the code from the command-line.

practicalli-johnny04:10:44

I also concur with Sean about the lein-ring plugin, it's quite simple to define a sever that can be managed by the repl in just a few lines of simple code https://practicalli.github.io/clojure-webapps/app-servers/simple-restart.html#code-example

practicalli-johnny04:10:34

Using this kind approach approach should make it easier to understand libraries like mount, integrant for managing the lifecycle of services (database, app server, etc)

seancorfield04:10:16

And if you do want to "look ahead" at some simple web apps that avoid lein-ring (in fact, avoid Leiningen altogether), that use Component, Compojure or Integrant, Reitit, along with some database access and and HTML templating, take a look at https://github.com/seancorfield/usermanager-example and https://github.com/PrestanceDesign/usermanager-reitit-integrant-example

suren06:10:02

Thanks guys I will keep these in mind.

Ping04:10:57

Is it normal to see a DevTools message i prod?

Ping04:10:06

Is it normal to see a DevTools message i prod? How can I resolve this?

DevTools failed to load SourceMap: Could not parse content 
for 
: 
Unexpected token < in JSON at position 0

Ping04:10:04

I also have this error. I am stuck here.

Ping04:10:01

Warning: React.createElement: type is invalid -- expected a 
string (for built-in components) or a class/function (for 
composite components) but got: undefined. You likely forgot 
to export your component from the file it's defined in, or 
you might have mixed up default and named imports.

Check the render method of `lba`.
in lba (created by mba)
in Suspense (created by M4)
in M4 (created by mba)
in mba (created by $$)
in $$
in Unknown (created by class_1)
in class_1

seancorfield05:10:47

@flora.makgaba "Unexpected token < in JSON" sounds like the HTTP request for the source map is returning an HTML page, but I don't know what "DevTools" you're referring to. If the source map isn't available, I'd expect a 404 Not Found error so perhaps your apps is not set up correctly? No idea about React. Depending on what specific tech you are using, there may be a more relevant channel to ask than here?

phronmophobic05:10:48

in #clojurescript

seancorfield05:10:33

Thanks. They also posted on StackOverflow (but no one has answered yet).

Mark Wardle14:10:11

Hi all. I am looking into configuration management - blending environmental variables, defined configuration files and system property files, perhaps command-line stuff - both properties (-D) and maybe arguments. There are many libraries here, which is fine for a single application, but what about when building libraries themselves? My initial thought on this, as a clojure beginner, is to build some kind of facade config pattern to decouple getting configuration data from the way it is set when used in a specific application? The facade simply presents configuration data as a map, obviously… but how that map is initialised would then depend on runtime - and not whether one uses mount or component or a specific config library or something else… A bit like tools.logging / slf4j facade and then a concrete implementation? Or is this a crazy thing and am I overthinking this?

practicalli-johnny12:10:24

I typically use aero for managing configuration https://juxt.pro/blog/aero

Mark Wardle21:10:46

Thanks. I did look at that, but one of the issues was having that as a dependency for something that’s logically a library - hence the thinking about facade pattern. But in retrospect, I think I was overthinking the whole thing, as its so trivially easy to switch. Thanks for the tip!

ghadi14:10:36

@mark354 you are correct to want a map as config. I would decouple the config data from the components themselves. The components should never "reach outwards" to discover config, config should be provided as an argument to the component's constructors

👍 3
Mark Wardle14:10:15

Thanks. That’s what I’ve done so far but even things like security credentials - feels messy once you have more than one or two configuration details for a specific function call.

ghadi14:10:14

elaborate?

ghadi14:10:41

another useful pattern is making config a first class system component

Mark Wardle14:10:43

Well… I have multiple modules that might have 4-5 configuration parameters that, in java, I’d use system properties to define. URLs for production, security credentials, etc.. Here’s some code as example:

(defn resolve!
  "Performs an EMPI fetch using the identifier as defined by `system` and `value` and
  the defined configuration.

  - url : the URL of the EMPI endpoint
  - processing-id : one of P (production) U (testing) or T (development) for type of server.
  - proxy-host : if required, proxy hostname as per clj-http
  - proxy-port : if required, proxy port as per clj-http"
  [system value {:keys [url processing-id proxy-host proxy-port] :as opts}]
  (-> (make-identifier-request system value opts)
      (do-post!)
      (parse-pdq)))

(deftype EmpiService [url processing-id opts]
  Resolver
  (resolve-id [this system value] 
    (resolve! system value (merge {:url url :processing-id processing-id} opts))))
So I’d much prefer to call out to some central configuration, but instead I’ve defined a type that can take those configurations and then allow a call to resolve! but that feels a bit OO-like and unnecessary… a style question really…

ghadi14:10:53

and providing the config component to your other components

ghadi14:10:16

heh, I used to work with EMPIs back in the day

ghadi14:10:32

you shouldn't care about whether your config comes from Java properties, environment variables, files, or S3

ghadi14:10:59

something should aggregate the application's config, and perform validation -- then provide the checked config map to things that need it

Mark Wardle14:10:35

That’s a good idea. Thanks.

Mark Wardle14:10:59

So I can isolate the config implementation, draw it together, and then inject into modules as necessary…

ghadi14:10:12

the config is a module itself

Mark Wardle14:10:45

So you used clojure in healthcare, or was that in a past life? 😉

ghadi14:10:54

yes and yes

👍 3
ghadi14:10:28

i first started using Clojure as an analyst at an academic medical institution

👍 3
Mark Wardle14:10:21

All my work in java until 3 years ago… halved the codesize and doubled my speed of development using golang… and now have done the same again moving to clojure, and yet get to use all my old libraries/ecosystem!

Mark Wardle14:10:41

Thanks for your help.

ghadi14:10:51

my pleasure

Matias Francisco Hernandez Arellano14:10:18

Hey folks.. I have a beginner question about maps. I have a api request that I do using clj-http.client and then parse the json with chesire So I get something like

(client/get url {:async? true} (fn [response] (let [items (get-items response)] ) )```

Matias Francisco Hernandez Arellano14:10:41

where get-itemsis just a wrapper around (:data (json/parse-string (:body response) true))

Matias Francisco Hernandez Arellano14:10:06

the issue comes after get-items this returns an array like structure with multiple items that have things like:

{ attributes: { title: '', company: { data: { attributes: { name: '' } } } }, links: { public_url: '' } }```

Matias Francisco Hernandez Arellano14:10:23

and I need to access all of that nested attribute.. what is the clojure way of doing it?

Matias Francisco Hernandez Arellano14:10:52

I know that I need to do something like (map parse-items items) and define parse-items

Matias Francisco Hernandez Arellano14:10:13

I was looking at select-keys but not sure how to used in this particular case

ghadi14:10:57

look at get-in

sova-soars-the-sora14:10:34

Hey @matiasfh nested data access can be done with clojure and there are also a couple of libraries invented to make this easier, one called Specter and another the name escapes me. Anyway, if I understand your question right, you want to traverse this nested structure? You can do it in pure clojure, other hats here can provide code, but you may want to check out Specter. There's a HN discussion about it from recently here: https://news.ycombinator.com/item?id=18578920

Matias Francisco Hernandez Arellano14:10:27

Yeah I want to traverse the array-like structure to get a few of the attributes of it so then i can use it for format a string that will be pushed back to output

Matias Francisco Hernandez Arellano14:10:35

will check the thread and Specter library

sova-soars-the-sora15:10:41

There's a really great comment in that thread that says "Specter solves a problem you should aim to avoid" x) but if you have no choice about the data coming in you may want it

ghadi15:10:00

I'm not a fan of giving beginners Specter

ghadi15:10:33

the marketing around Specter is hyperbolic

ghadi15:10:24

for a #beginner, focus on learning the core library

Matias Francisco Hernandez Arellano15:10:50

Yeah.. in fact I want to learn the clojure way. And @sova yes.. I can't fix the way the data is returned by the api is a 3rd party server

Matias Francisco Hernandez Arellano15:10:26

The goal of this is to retrieve some of the attributes of this structure to create a set of strings based on that attributes

dpsutton15:10:26

so you already know how to map. and your question is how to define a function parse-item which works on the following map:

{:attributes {:title "title"
              :company {:data {:attributes {:name "company name"}}}}
 :links {:public-url "public url"}}

dpsutton15:10:58

given this map, what would you like parse-item to return?

Matias Francisco Hernandez Arellano15:10:06

the map is bigger.. will have more items inside the attributes level so what I want is to map an get a easier structure like

{:title: "title", :companyName: "some", 🔗 "public_url" }```

Matias Francisco Hernandez Arellano15:10:58

I use to do this with JS by just mapping over the array of objects to create a new array of objects with the new shape.. but that is the JS way.. i figure that is somehow similar with Clojure in terms of the approach. map the map

dpsutton15:10:21

(let [data {:attributes {:title "title"
                         :company {:data {:attributes {:name "company name"}}}}
            :links {:public-url "public url"}}]
  {:title (get-in data [:attributes :title])
   :company-name (get-in data [:attributes :company :data :attributes :name])
   :link (get-in data [:links :public-url])})

dpsutton15:10:38

shows how to get this data

dpsutton15:10:07

so you could do

(defn parse-item [data]
  {:title (get-in data [:attributes :title])
   :company-name (get-in data [:attributes :company :data :attributes :name])
   :link (get-in data [:links :public-url])})

(map parse-item response)

Matias Francisco Hernandez Arellano15:10:26

you know what.. I had that in mind but was unsure of the idea of how that is written.. Thanks!

👍 6
sova-soars-the-sora15:10:18

Looking good. and as @U050ECB92 says it's sage advice to learn your way around core 😃 many many useful scissors, snips, and pocket knives in there.

💪 6
Santiago16:10:24

In which situations would (:key coll) return nil if coll is a map? and :key is def not nil because (vals coll) returns the non-nil values and (keys coll) returns :key

Santiago16:10:54

If anyone is interested this is my full script. I’m scratching my head as to why the final lines don’t give me the value of :comments for each map in the vector. (map map?) is true, (map vals) gives me the correct values.. 🤷

Santiago16:10:05

#!/usr/bin/env bb

(ns jcpsantiago.controltower.airportsscript
  (:require [babashka.curl :as curl]
            [clojure.string :as s]
            [babashka.pods :as pods]))


(pods/load-pod "bootleg")
(require '[pod.retrogradeorbit.bootleg.utils :refer [convert-to]]
         '[pod.retrogradeorbit.hickory.select :as hs])


;; Airline data ;;
(def wiki-airlines 
  (-> ""
      curl/get
      :body))


(defn deepest-text
  "Drill down to the deepest text node(s) and return them as a string."
  [node]
  (cond (vector? node) (apply str (mapcat deepest-text node))
        (map? node) (deepest-text (:content node))
        :else node))


(defn extract-tables [html]
  "Takes an html page and extracts a table element."
  (mapv (fn [table]
          (mapv #(mapv deepest-text
                       (hs/select (hs/or (hs/tag :th) (hs/tag :td)) %))
                (hs/select (hs/tag :tr) table)))
        (->> (convert-to html :hickory) 
             (hs/select (hs/tag :table)))))


(defn to-keyword
  "This takes a string and returns a normalized keyword."
  [input]
  (-> input
    s/lower-case
    (s/replace \space \-)
    (s/replace "/.+" "")
    keyword))


(defn html->map 
  "Extracts an html table and turns it into a map."
  [wiki-airlines] 
  (let [table (first (extract-tables wiki-airlines))
        headers (->> table
                     first
                     (map to-keyword)
                     vec)
        rows (->> table
                  (drop 1)
                  (mapv #(mapv s/trim-newline %)))]
    (println "Joining rows and headers...")
    (map #(zipmap headers %) rows)))


(defn complete?
  [k coll]
  (every? true? (map not [(= (k coll) "\n")
                          (= (k coll) "n/a")])))

(defn not-defunct?
  [coll]
  (let [c (s/lower-case (:comments coll))]
    (println c)
    (not (s/includes? c "defunct"))))


(def complete-icao? (partial complete? :icao))
(def complete-iata? (partial complete? :iata))


(->> (html->map wiki-airlines)
     (filter (every-pred complete-iata? complete-icao?)) 
     (map :comments))

dpsutton16:10:38

does it return :key: (with a trailing colon)?

Santiago16:10:20

no sorry, that’s a typo:sweat_smile:

hiredman16:10:48

what you can do is (doseq [[k _] some-map :when (= (vec (.getBytes (name k))) (vec (.getBytes (name :key))))] (prn k))

hiredman16:10:52

which will print out true if any keyword key in the map has a name that is byte for byte the same as the name of your :key

hiredman16:10:01

er, print out the key

hiredman16:10:25

if it doesn't the keyword you are using as a key in your map (by calling keyword on data you pull from the internet) contains characters that aren't printing, either because of your terminal setup, or because they are some kind of funky whitespace

hiredman17:10:07

you are stripping out the space character, but there are many other kinds of whitespace

Santiago17:10:25

interesting… saving to a text file as edn and reading it back works fine. Wouldn’t this also carry over any whitespace characters?

Santiago17:10:19

I saved it using pr-str and spit

Santiago17:10:57

(->> (html->map wiki-airlines)
     pr-str
     edn/read-string
     (map :icao))
works fine, is this a sign of something strange happening with character encoding @U0NCTKEV8?

hiredman17:10:31

no idea what works find means

hiredman17:10:54

ah, I see, no, my guess is there is some other kind of whitespace character that is not a space

hiredman17:10:40

so your :icao keywords all contain an extra character that is not visible when printed, which means it doesn't round trip

Santiago19:10:08

seems like that ws the issue.. string/trim fixed it geez.. thanks!

Karo16:10:24

Hi, what tool can be used to automate creation and update of openAPI documentation in clojure?

Tzafrir Ben Ami18:10:47

if you’re using https://github.com/metosin/reitit, it comes with “built it” swagger support. See an example: https://github.com/metosin/reitit/tree/master/examples/http-swagger

👍 3
Karo07:10:52

Thank you

sova-soars-the-sora17:10:51

great question! 😃 bump.

João Galrito18:10:44

hello, why is the output of this:

(defn create-es-bulk-writer []
  (let [es-write-chan (async/chan)]
    (println 1)
    (async/go
      (println 2))
    (println 3)
    es-write-chan))
this:
1
3

João Galrito18:10:46

default thread pool is full (and blocked)?

marshall18:10:42

put a do around the printlns:

marshall18:10:47

(defn create-es-bulk-writer []
  (let [es-write-chan (async/chan)]
    (do 
     (println 1)
     (async/go (println 2))
     (println 3) )
    es-write-chan))

marshall18:10:58

user=> (create-es-bulk-writer )
1
3
#object[clojure.core.async.impl.channels.ManyToManyChannel 0x6c26e588 "clojure.core.as2y
nc.impl.channels.ManyToManyChannel@6c26e588"]
user=>2

João Galrito18:10:27

i think the problem was threads

João Galrito18:10:32

i restarted the repl and now it works

marshall18:10:50

that said, you’re not actually using the channel you make

João Galrito18:10:09

i changed the code before posting

João Galrito18:10:12

but left some stuff

João Galrito18:10:23

the go block was waiting on the channel

João Galrito18:10:53

and I was using "stop execution" on the repl but I guess the go threads were still running waiting for messages on the channel

Artem19:10:58

Hello, I have more than 100 files in my lein project that describe the tasks, the data for them and their solution to make sure that the solution meets the described data I am testing the call of the solve function in another test file. For example, I have namespace task-one and task-two and files task_one.clj and task_two.clj, to test them I need to create a separate task_one_test.clj and task_two_test.clj inside test directory, inside test files the same test function is launched, only test data and solution functions differ. Since the task files are more than 100, the test files are also more than 100, but I want to run tests for all the tasks inside one common file, without having to create additional files. I have understood that the deftest macro is tied to the namespace in which it runs, so the test will run only once for this common file, which does not solve my problem with a large number of files describing the same tests. How this problem can be solved?

andy.fingerhut22:10:24

I am not sure I followed all of your description, but you can put all of the tests for a project into a single file of code if you want, in a single namespace, with as many separate deftest forms as you want (or as few as one, but having separate deftest forms can make it easier later to run only a subset of the tests.

andy.fingerhut22:10:08

No matter how many namespaces your production code is spread across, a single test file can call functions in all of them. There is no requirement that task_one.clj is tested by code in a file named task_one_test.clj

Artem23:10:31

My main goal is to get rid of the heap of files and avoid copying the deftest to each task, I would like to run tests dynamically for each task, for example by getting a list of namespaces and run a function like this for every task:

(defn run-test [result data solution task-name]
  (deftest task-name
    (is (= result (apply solution data)))))
but for now I can’t figure out how to do it 😞

practicalli-johnny12:10:20

If it is the assertions, eg. is expressions, that are repetative, the are function can streamline tests https://practicalli.github.io/clojure/testing/unit-testing/writing-unit-tests.html#testing-assertions-with-a-data-set

👍 3
Artem23:10:09

Thanks for the link! Solved my problem, that was more easy than I thought!