Hey all, I'm trying to host a simple Reagent and Re-Frame project on github's gh-pages, but it's not working for me

Here's the repo: https://github.com/JimTheMan/Kablamo

This is the live site link: https://jimtheman.github.io/Kablamo/

I'm getting the error "Failed to load resource: the server responded with a status of 404 () http://jimtheman.github.io/:11 Uncaught ReferenceError: kablamo is not defined at http://jimtheman.github.io/:11"

btw kablamo is the name is the whole project (scaffolded it with 'lein new re-frame kablamo")

I just ran lein cljsbuild once min and pushed the stuff in kablamo/resources/public to the branch named gh-pages

Seems to be working for me @derpocious -- what am I supposed to see?

there should be yellow and red lines moving around forming some text

For me it just shows a blank blue page

and the errors in the console

I see the moving text -- and the re-frisk debugger pop-out.

And that's with Microsoft Edge on Windows 10 so it's not like the bar is very high... 🙂

hmm I just tried on a different computer, and that one is showing the errors also

just a blank blue page

anyway, I have to go to sleep. will try more tomorrow. thanks

it works for me - just a warning, it is trying to connect to figwheel on my localhost, and while I'm not certain I could mess with your server by connecting to the app via figwheel, I'm also not certain I couldn't

What is the relationship between new Clojure Deps and other dependendency management tools like Lein? Are they complementary or do I still have to use only project.clj to define the dependencies? https://clojure.org/guides/deps_and_cli

Because I really like featues like referencing dependency directly from Github however I don't understand from the article if I can use it directly in my project or only for REPL development purposes

@petr.mensik tools.deps is a low-level library that other tools can build on top of. In the Boot world we have https://github.com/seancorfield/boot-tools-deps that brings pretty much all the (readonly) functionality of tools.deps to Boot.

I think there are efforts to produce Leiningen plugins to do all this but they are not so far along yet.

The main difference is that clj and tools.deps are about running Clojure code, not building artifacts etc. Boot (and Leiningen) are about building and deploying artifacts.

@seancorfield Thanks for clarification 🙂

I'm constantly surprised at the questions around tools.deps but I guess the Clojure/core folks are just failing to communicate what its purpose really is... It doesn't replace lein / boot.

@seancorfield I understand that tools.deps is not a build tool however I didn't quite grasp how it complements with dependency management tools we already have

There is no mention of Leiningen nor Boot in the article so I guess that's what causes the confusion (at least it caused it in my case 🙂 )

Yeah, I think maybe @alexmiller needs to think about the message around that?

We've been using deps.edn files at work for ages (albeit in a different format) so tools.deps has just seemed like a natural adjunct to existing build tools to me...

...but I guess we're not "typical" 🙂

I wonder what to do with leiningen if you have a build with dependencies on github. Most likely they will be included in the sources I think? In oder for other projects depending on it not getting them also from github?

Can someone help me understand how this works?

(def line "foo:bar")
           (clojure.string/split line #":") ;; => ["foo" "bar"]
             (into {}   (clojure.string/split line #":")) ;; =>    java.lang.Character cannot be cast to java.util.Map$Entry 
             (into {}   [(clojure.string/split line #":")]) ;; => {"foo" "bar"}

to be more specific, I don't understand why the second doesn't work and the third does

2nd form is trying to put strings into a map instead of trying to put kv pairs in

into {} expects a list of lists, with the inner set of lists each having 2 items corresponding to the key and the value

ah, ok. didn't realize that it was expecting a list of lists. that makes sense

into is essentially (reduce conj coll1 coll2), and as such operations on collections

while vectors are also collections, in the case of putting them into a map, they are single values

I’m over-simplifying a bit, but that’s the specific case you’re seeing anyway.

so you need the vectors for the k/v of the map, and then a collection around that for the reduce

yeah i guess it wouldn't make much sense for it to take just a vector, since it could have an arbitrary number of elements instead of 2

thanks for your help!

@nickstares0 Note, it's not a "list of lists" it's a "coll" of "vectors", if you did a (into {} [(seq (str/split "foo:bar" #":"))]) it would also fail. It works since str/split returns a vector

i have a lein project, and it was working fine yesterday, but now when i do lein run, it just hangs. it prints one line that says: 2018-01-29 10:26:58,081 [main] DEBUG org.jboss.logging - Logging Provider: org.jboss.logging.Slf4jLoggerProvider but nothing happens and if i refresh my browser it says it can’t connect to the server. what could be causing this?

@feihong.hsu It looks like something is getting stuck during server start-up. Maybe you have an invalid configuration setting somewhere? What changed since yesterday?

Can someone help me? What would be the idiomatic way to do some action depending on which key a map has? I have a list of maps like [{ :x 1 } {:y 2}] and I want to do something if the map has the first structure and something different for the other. I can have many different structures.

@manutter51 that’s the weird thing. i haven’t changed any configuration settings in more than a week (that i know of). when i create a new leiningen project, it’s able to start up just fine. but if i did have an invalid configuration setting, wouldn’t it error out instead of just hanging?

@feihong.hsu I dunno, it’s kind of hard to tell. Do you know what changed since the last time it ran correctly? Have you done a lein clean lately?

do ctrl-\ to dump your stack and see where it’s hung (or if it’s a separate process you can kill -3 the pid)

or use jstack

@manutter51 so i finally found the problem but it wasn’t a change in configuration. it was an additional require that i put in env/dev/clj/user.clj. at first i thought it was a problem with circular imports, but i remember that when i did that before, i got an error message. the other weird thing is that i have another source file that has very similar imports and that one doesn’t cause the app to hang. happy to provide a link to my source code on github if that would help.

@manutter51 argh, ignore the above. it doesn’t have anything to do with require. i was making a call to a function inside user.clj, but that function was not able to complete for some reason.

yesterday I’ve asked about sending message between producer/consumers using channels. finally I could manage and was able to solve https://gist.github.com/tdantas/4d172af3451054290dc67541f5f71d60 cheers and thx guys

Hey guys how can I count key-value to a new object some like this: input -> {:type1 [{:type type1}] :type2 [{:type type2} {:type type2} {:type type2}]} output -> {:type1 1 :type2 3}

(map (fn [base_item]
                       (prn "TEST")
                       (map (fn [target_item]
                              (prn "TEST")
                              (when (match-percent-logic base_item
                                                         target_item)
                                [base_item target_item]))
                            items))
                     items)

`java.lang.NullPointerException: null Reflector.java:26 clojure.lang.Reflector.invokeInstanceMethod /Users/jtackett/Desktop/Foodbam/food_matching/src/food_matching/match.clj:145 food-matching.match/match-percent-logic /Users/jtackett/Desktop/Foodbam/food_matching/src/food_matching/match.clj:107 food-matching.match/match-percent-logic /Users/jtackett/Desktop/Foodbam/food_matching/src/food_matching/match.clj:183 food-matching.match/matching-engine[fn] core.clj:2646 clojure.core/map[fn] LazySeq.java:40 clojure.lang.LazySeq.sval LazySeq.java:49 clojure.lang.LazySeq.seq RT.java:521 clojure.lang.RT.seq core.clj:137 clojure.core/seq core_print.clj:46 clojure.core/print-sequential`

what am I messing up here?

the error is inside match-percent-logic

@noisesmith but first line in match-percent-logic is a prn and I don't see the prn

also, FYI, a nested map can usually be replaced with for (for [x items y items :when (match-percent-logic x y)] [x y]) *(fixed the binding names to match)

Ah very cool

@josh_tackett the stack trace says that match-percent-logic was called

strange ok

I'll look deeper

thank you

@noisesmith why am I seeing my prn's in the output of the map?

(("Starting match algorithm . . . " "Item 1 > > > " <result> . . .

because map is lazy

the prints happen as the items are realized

ah

if you capture the output of map, you can make more sense of it

eg (def foo (doall ...)) (now all the printlns are done) (pprint foo)

(doall 
                  (map (fn [base_item]
                         (map (fn [target_item]
                                (when (match-percent-logic base_item
                                                           target_item)
                                  [base_item target_item]))
                              items))
                       items))

like so ^^^ ?

a favorite trick of mine is to use data side effects instead of i/o side effects

right but inside a def

because the def will just return nil

(let [
        _ (prn "Items - > " items)
        ;; Now do the N^2 checking to see if the matches...well...match
        matches (doall 
                  (map (fn [base_item]
                         (map (fn [target_item]
                                (when (match-percent-logic base_item
                                                           target_item)
                                  [base_item target_item]))
                              items))
                       items))

        ]
    matches)

still seeing the prn's

that still doesn't fix it

nvm go it

you probably need a nested doall

just using mapv and mapv

sure, that's another way to accomplish it

anyway, what I was saying about capturing data side effects rather than doing i/o side effects (defonce debug (atom [])) (map (fn [x] (swap! debug conj {:x x}) (f x)) items)

then you can programatically access and work with the values being captured, instead of just reading them on the screen

@noisesmith ok now we're talking, was just about to ask you how to speed up this algo so its not N^2

clojure.math.combinatorics might help there

What's that for?

https://github.com/clojure/math.combinatorics

for efficiently generating combinations of an input

I don't think applies here

I'm basically doing millions of string comparisons with a nested map

a nested map is a naiive way to get all combinations of two items in an input

if you only want distinct combinations, or only want combinations where the two items are not the same, there's optimizations possible

No I'm looking at fuzzy string comparison

@noisesmith ^^^

so basically the fastest way to see which items in the list match each other via a fuzzy string comparison

I'm talking about the wrapping context, that's the thing that is n^2

you map items inside map items, which means you are comparing every pair of elements in items

if the thing you want to optimize isn't in the code you've shared I can't help much with that unless you show more

you could use a memoized function for the comparison, so that you never calculate the same comparison twice

depending on whether you expect the data to repeat

all I need to optimize is what I've shared @noisesmith so maybe I'm just confused

what is best way to optimize nested (map (map ...) ...)?

i think his point is that the only way you can optimize map map is to do less work. and to do less work you need to know what you are doing and figure out a way to prune the search space. but all we know is that you need to compare all strings. and if so, there's no short cut around that

😞

well, the shortcut could be eliminating duplicate comparisons, or caching the comparison function with memoize

but you need more context to know for sure

@noisesmith maybe I can make some of the process async to speed it up?

you could parallelize sure

how would you parallize that loop?

check out the reducers namespace - there's probably something there that's optimal https://clojure.org/reference/reducers

in fact, there's drop-in replacements for map in there iirc

looks like what you would want is fold with the map transducer

(or maybe r/map which isn't a transducer (precedes those) but has similar purpose)

so just swap map for r/map ?

ah wait I see

just wrap it in fold

not quite - you need to put it inside a fold, which needs a reducing function

exactly!

the reducing function would basically be a conj - also your when can become r/filter I think

[#object[clojure.core.reducers$folder$reify__6593 0x67999bad "clojure.core.reducers$folder$reify__6593@67999bad"]

(mapv (fn [base_item]
                        (mapv (fn [target_item]
                                (r/filter (match-percent-logic base_item
                                                               target_item)
                                          [(:din base_item)
                                           (:din target_item)]))
                              items))
                      items)

so now I'm getting objects, do I just need to deref them?

r/filter needs to be inside r/fold

I was saying that once you use r/fold and r/map, you can also use r/filter and simplify that inner logic

(r/map (fn [base_item]
                        (r/fold
                          conj
                          (r/map (fn [target_item]
                                  (r/filter (match-percent-logic base_item
                                                                 target_item)
                                            [(:din base_item)
                                             (:din target_item)]))
                                items)))
                      items)

r/map has to be inside r/fold

hahaha should have known

should I just wrap the outer loop in fold?

or both maps?

I'd take the time to read the clojure guide on reducers - I haven't used them seriously yet, but your use case looks like a good one for reducers

(r/fold
                   conj
                   (r/map (fn [base_item]
                            (r/fold
                              conj
                              (r/map (fn [target_item]
                                       (r/filter (match-percent-logic base_item
                                                                      target_item)
                                                 [(:din base_item)
                                                  (:din target_item)]))
                                     items)))
                          items))

so that works, but the response is a bunch of objects

rather than the actual result

@noisesmith maybe I should move this to actual clojure channel?

maybe? these are still beginner questions, but use your best judgment - I think some of the examples in the "Using Reducers" section on the clojure guide I linked to look a lot like your use-case

(r/fold
                   conj
                   (r/map (fn [base_item]
                            (r/fold
                              conj
                              (r/map (fn [target_item]
                                       (r/filter (match-percent-logic base_item
                                                                      target_item)
                                                 [(:din base_item)
                                                  (:din target_item)]))
                                     items)))
                          items))

so I agree this is a great lib

but nothing in there says how to deref after

it just shows the result...well...working haha

Using Reducers shows it

not really though right?

(require '[clojure.core.reducers :as r])
(r/fold + (r/filter even? (r/map inc [1 1 1 2])))
;=> 6

it shows this

but that's not very different from what I have

and they just show the reuslt

result

it looks like either (into [] ...) or (r/foldcat ...)

nah still objects

r/map might not work directly in the nested way - you might need an inner foldcat, into, or fold

(r/foldcat
                   (r/map (fn [base_item]
                            (r/foldcat
                              (r/map (fn [target_item]
                                       (r/filter (match-percent-logic base_item
                                                                      target_item)
                                                 [(:din base_item)
                                                  (:din target_item)]))
                                     items)))
                          items))

but they have a nested map in their example?

foldcat is just the same as (r/fold conj ....)

So just got it working, but ironically its not any faster hahaha

hmm... well it's implicitly parallelizing supposedly - what was the final thing that made it work?

I removed the r/filter and got an actual result

I'm going to try it with more items and see the difference in speed

let's see what they say in #clojure @noisesmith

Also @noisesmith do you know a way to make sure that loops don't return a bunch of nils but rather just skip the result?

[["28622" "28622"] nil nil nil nil nil nil nil nil nil]

this is forcing me to go through the list again

you can use (vec (keep ...)) instead of mapv

or use for which takes a :when syntax

if you are OK with [[a b] nil] turning into [a b] you can use mapcat instead of map

or (into [] (mapcat ...) coll)

I would suggest breaking this thing apart @josh_tackett... too much going on in one expression

get the inner part working, give it a name, get the other layers working

@ghadi yes after we get it working I'll obviously make it more modular

that is backwards

ok

it's not working because it isn't decomposed

and you don't know where the error is

its actually work now 😉

whew

sometimes its easier to see the function all together rather than trying to follow through many defn's

@noisesmith that reducer lib was a big help thank you