Fork me on GitHub
#beginners
<
2016-08-07
>
senya2204:08:20

Say I have the following structure [#{:e :c :d} #{:f}]

senya2204:08:56

I'd like to express it as a dependency graph via (require '[com.stuartsierra.dependency :as dep])

senya2204:08:15

kind of like

(def graph1 (-> (dep/graph)
                (dep/depend :f :e)
                (dep/depend :f :c)
                (dep/depend :f :d)))

senya2204:08:56

how would I express it in a function?

senya2204:08:14

I tried this, but I'm not too sure it does what I'm after:

senya2204:08:30

(defn build-rule-tree [graph rule]
  (let [[lhs rhs] rule]
    ;(prn lhs :-> rhs)
    (map #(-> graph (dep/depend rhs %)) lhs)
    
    ))

senya2204:08:53

(build-rule-tree (dep/graph) [#{:b :a} #{:c}])

senya2204:08:32

is there a way to keep a global graph object as I need to walk a set of vectors, say

([#{:b :a} #{:c}] [#{:e :c :d} #{:f}] [#{:e :k :d} #{:m}] [#{:t :p} #{:k}])
eventually to build the whole graph

porglezomp14:08:31

Isn’t that what reduce is for?

porglezomp14:08:41

(reduce build-rule-tree dep/graph ([#{:b :a} #{:c}] [#{:e :c :d} #{:f}] …))

porglezomp14:08:58

Unless I’m misunderstanding something

senya2216:08:55

Perhaps... I'm trying to implement something similar since, but getting exceptions:

(def rule1 [[:a :b] :-> :c])
(def rule2 [[:c :d :e] :-> :f])
(def rule3 [[:k :d :e] :-> :m])
(def rule4 [[:p :t] :-> :k])

(def ruleset [rule1 rule2 rule3 rule4] )

(defn set-rule-parser [rule]
  (let [[inputs _-> & outputs] rule]
    [(set inputs)
     (set outputs)]))

senya2216:08:32

(defn break-out-rules [ruleset]
  (map set-rule-parser ruleset))

(defn task2 [ruleset]
  (let [into-sets (break-out-rules ruleset)]
    (println "into-sets: " into-sets)
    (reduce graph-building-reducer dep/graph into-sets)
    ))

(defn graph-building-reducer [[lhs rhs]]

  (map #(dep/depend rhs lhs) lhs)

  )

senya2216:08:02

(defn graph-building-reducer [[lhs rhs]]

  (map #(dep/depend rhs lhs) lhs)

  )
CompilerException java.lang.IllegalArgumentException: No single method: depend of interface: com.stuartsierra.dependency.DependencyGraphUpdate found for function: depend of protocol: DependencyGraphUpdate, compiling:(C:\Users\Simeon\AppData\Local\Temp\form-init1745806617618307544.clj:3:9) 

senya2216:08:59

Not sure what's wrong. And after changing:

(defn graph-building-reducer [[lhs rhs]]

  (map #(prn rhs "->" lhs) lhs)

  )
it complains about something else:
(task2 ruleset)
IllegalStateException Attempting to call unbound fn: #'task2.core/task2  clojure.lang.Var$Unbound.throwArity (Var.java:43)

eggsyntax16:08:30

You can also do extremely powerful data structure building & transforming with clojure.zip (https://clojure.github.io/clojure/clojure.zip-api.html) but it’s a steeper learning curve & quite possibly overkill for what you’re doing.

eggsyntax16:08:51

There’s also the 3rd-party specter lib (https://github.com/nathanmarz/specter), which is a really powerful way to do nested data structure transforms.

eggsyntax16:08:16

That’s just with a very quick look at what you’re doing; I could be misunderstanding what you’re after.

eggsyntax16:08:43

Alternately, if you just want to express rules and check data against them, without explicitly building a dependency graph, core.logic is a fantastic tool for that. That’s not the way you’ve expressed the problem, but depending on what you’re going to use the dependency graph for, it may be another approach.

senya2217:08:26

Briefly - I'm writing a function that performs basic inference on rule systems with the rules expressed as shown above. It should take two arguments: a set of rules and a set of input assertions

eggsyntax17:08:57

And you want to check to see whether the assertions are valid given the set of rules? core.logic may be the ideal tool for that, then.

senya2217:08:39

At this stage, the function it should return the set of all outputs that the rule system can derive from those inputs

senya2217:08:59

At a later stage, this function is going to be modified to take a set of rules as its only argument, uses them to create a pretreated data structure somehow, and returns a new function as its result. This resulting function should take a set of input assertions as its only argument, and it should return the set of valid inferences with the aid of what was computed in the pretreatment step.

eggsyntax17:08:05

Yeah, I’d take a look at core.logic for that. In some complex cases (eg if you may have rules that result in infinite loops, say if you had rules :a -> :b and :b -> :a, then you might need to consider a rules engine. But them’s the big guns 😉

eggsyntax17:08:00

Glancing back, I see you’ve already been going with the current approach for a while — don’t let me deter you from that! Depends, I guess, whether you’re doing it as a learning exercise or you just want results 😄. With your current approach, walk and specter seem likely to come in handy.

senya2217:08:06

Results, of course 🙂 And learning - must be obvious 🙂

eggsyntax17:08:22

You might also want to consider using core.match to express your ruleset. https://github.com/clojure/core.match

senya2217:08:15

I looked into that - I didn't quite grasped what it is for yet.

senya2217:08:58

That's why I tried this reducing approach - would like to figure why the graph-building-reducer breaks

eggsyntax17:08:21

I’m not familiar enough w/ Sierra’s dependencies lib to have any sense of why you’re getting that error.

eggsyntax17:08:36

In general, the error you’re getting (as of 12:17) means that dep/depend is a protocol function, but doesn’t have a matching arity. You’re calling it with the args rhs and lhs — what args does its documentation say it wants?

senya2217:08:23

I tried eliminating that library and it threw the other error that I listed (as of 12:36) - I'm suspecting it's my syntax that is wrong

eggsyntax17:08:41

Hmm, at a glance it does look like it has a 2-arg arity, based on the examples in the readme.

eggsyntax17:08:29

Although in the code it looks like it wants 3:

(defprotocol DependencyGraphUpdate
  (depend [graph node dep]
(from https://github.com/stuartsierra/dependency/blob/master/src/com/stuartsierra/dependency.cljc )

senya2217:08:58

I tried the example from that site, it works fine:

(def g1 (-> (dep/graph)
            (dep/depend :b :a)   ; "B depends on A"
            (dep/depend :c :b)   ; "C depends on B"
            (dep/depend :c :a)   ; "C depends on A"
            (dep/depend :d :c))) ; "D depends on C"

eggsyntax17:08:36

Ah, but in that case, the 1st argument is being inserted by the threading macro (`->`).

eggsyntax17:08:44

ie (-> (dep/graph) (dep/depend :b :a)) is equivalent to (dep/depend (dep/graph) :b :a)

senya2217:08:51

Yes, looks like we can do that through a reduce as per @porglezomp 's suggestion

senya2217:08:11

which I was trying to implement

senya2217:08:57

Something is with my reducing function I think

eggsyntax17:08:58

Maybe instead of (map #(dep/depend rhs lhs) lhs) you meant (map #(dep/depend rhs %) lhs)?

senya2217:08:41

makes sense, let me try it

eggsyntax17:08:59

But then you still need to have a way to insert the graph arg in there. Or did you perhaps mean (map #(dep/depend % rhs lhs) lhs)? That only makes sense if you’re treating the lhs as the graph.

senya2217:08:29

well, the into-sets is of this form:

into-sets:  ([#{:b :a} #{:c}] [#{:e :c :d} #{:f}] [#{:e :k :d} #{:m}] [#{:t :p} #{:k}])

senya2217:08:40

so I want to say that the second(rhs) set of each vector depends on every element in the lhs set

senya2217:08:46

As for inserting graph, I'm not sure whether I need to pass it in a call to reduce, or should it be mentioned as a first arg in the reducing function

eggsyntax17:08:53

(map somefn lhs) means that you want to apply somefn to each member of lhs. Is that what you're meaning to do? Or is it that you want to apply it to each element in the into-sets list?

porglezomp17:08:53

Let me write up a snippet here...

senya2217:08:24

the former, I think:

senya2217:08:31

(defn graph-building-reducer [graph [lhs rhs]]

  ;(map #(prn rhs "->" lhs) lhs)
  (map #(dep/depend graph rhs %) lhs)

  )

eggsyntax17:08:12

OK, in that case the expression you just gave seems correct.

senya2217:08:29

this doesn't complain, however it does when I'm trying to invoke:

=> #'task2.core/graph-building-reducer
(task2 ruleset)
IllegalStateException Attempting to call unbound fn: #'task2.core/task2  clojure.lang.Var$Unbound.throwArity (Var.java:43)

eggsyntax17:08:14

OK, I'm gonna have to take a more thorough look at the code and get a more general sense of what's going on. (unless @porglezomp clears it up for you in the meantime)

porglezomp17:08:49

Will it always be [#{:b :a} #{:c}], or might it be [#{:b :a} #{:c :d}]?

porglezomp17:08:06

If it’s the former why not write them as [#{:b :a} :c]?

porglezomp17:08:19

I’m just asking so I can make the write assumptions in this snippet here.

senya2217:08:25

Hm, I thought about it. It seems that the consequence of the rule is always a singular outcome the way the problem is stated

senya2217:08:42

So the set-rule-parser handles creating of the vector of sets - something that was built in a previous step.

porglezomp17:08:04

Okay, so you’re being handed this set by something else. I’ll work for a set of one element only?

senya2217:08:18

That previous step also used to reduce the resulting structure into-set, but differently, but I think you were right by suggesting that reduce could be used here as well - that's why I went this route

senya2217:08:10

Let me copy the whole set of defns once again

senya2217:08:56

(def rule1 [[:a :b] :-> :c])
(def rule2 [[:c :d :e] :-> :f])
(def rule3 [[:k :d :e] :-> :m])
(def rule4 [[:p :t] :-> :k])

(def ruleset [rule1 rule2 rule3 rule4] )

senya2217:08:56

(defn task2 [ruleset]
  (let [into-sets (break-out-rules ruleset)]
    (println "into-sets: " into-sets)
    (reduce graph-building-reducer dep/graph into-sets)
    ))

(defn graph-building-reducer [graph [lhs rhs]]

  ;(map #(prn rhs "->" lhs) lhs)
  (map #(dep/depend graph rhs %) lhs)

  )

senya2218:08:14

(defn break-out-rules [ruleset]
  (map set-rule-parser ruleset))

senya2218:08:37

(defn set-rule-parser [rule]
  (let [[inputs _-> & outputs] rule]
    [(set inputs)
     (set outputs)]))

senya2218:08:54

(task2 ruleset)

senya2218:08:08

(require '[com.stuartsierra.dependency :as dep])

senya2218:08:58

So this is the task2.

senya2218:08:37

And before, here's how the reducing was done, since we needed just 2 resulting collections - all inputs and all outputs

senya2218:08:50

;;defs
(def rule1 [[:a :b] :-> :c])
(def rule2 [[:c :d :e] :-> :f])
(def rule3 [[:k :d :e] :-> :m])
(def rule4 [[:p :t] :-> :k])

(def ruleset [rule1 rule2 rule3 rule4] )

;;task 1
(require '[clojure.set :as set])

(defn set-rule-parser [rule]
  (let [[inputs _-> & outputs] rule]
    [(set inputs)
     (set outputs)]))

(defn reducer [[accumulated-input
                accumulated-output]
               [new-input
                new-output]]
  [(set/union accumulated-input new-input)
   (set/union accumulated-output new-output)])

(defn break-out-rules [ruleset]
  (map set-rule-parser ruleset))

(defn task1 [ruleset]
  (let [into-sets (break-out-rules ruleset)]
    (println "into-sets: " into-sets)
    (reduce reducer into-sets)
    ))

(task1 ruleset)
;into-sets:  ([#{:b :a} #{:c}] [#{:e :c :d} #{:f}] [#{:e :k :d} #{:m}] [#{:t :p} #{:k}])
;=> [#{:e :k :c :b :d :t :p :a} #{:m :k :c :f}]

porglezomp18:08:51

Mine works on the original ruleset, does this do what you want?

eggsyntax18:08:30

@porglezomp may have a complete solution for you; I'm just pointing out what was wrong with the code you had written. I'll turn it over to y'all (but feel free to tag me w/ questions).

porglezomp18:08:36

I don’t have much more to say than what I have there, I’m a beginner to Clojure but a veteran at using reduce.

eggsyntax18:08:49

Actually I screwed that up. Editing.

eggsyntax18:08:15

(failed to notice you still had the map in graph-builder-reducer & made a mess of things). Here's what I think you meant to do:

eggsyntax18:08:18

(defn graph-building-reducer [graph [lhs rhs]]
  (dep/depend graph rhs lhs))

(defn task2 [ruleset]
  (let [into-sets (break-out-rules ruleset)]
    (println "into-sets: " into-sets)
    (reduce graph-building-reducer (dep/graph) into-sets)))

senya2218:08:05

@porglezomp: Mindboggling - I need to try to see what it returns 🙂

eggsyntax18:08:08

That gives the following result (let me know if it matches your intention):

into-sets:  ([#{:b :a} #{:c}] [#{:e :c :d} #{:f}] [#{:e :k :d} #{:m}] [#{:t :p} #{:k}])
{:dependencies {#{:c} #{#{:b :a}}, #{:f} #{#{:e :c :d}}, #{:m} #{#{:e :k :d}}, #{:k} #{#{:t :p}}},
 :dependents {#{:b :a} #{#{:c}}, #{:e :c :d} #{#{:f}}, #{:e :k :d} #{#{:m}}, #{:t :p} #{#{:k}}}}

porglezomp18:08:57

dep/depend needs to be run on each element respectively though, doesn’t it? Not (dep/depend graph #{:a :b} #{:c}) but (-> graph (dep/depend :c :a) (dep/depend :c :b))

eggsyntax18:08:54

Ah, ok, gotcha.

eggsyntax18:08:47

I see, so you solved it w/ the additional reduce step. Seems sound to me 😄

senya2218:08:24

Tried @eggsyntax's snippet - seems to be right, at the same time @porglezomp is right, it needs to thread onto each element

porglezomp18:08:49

Does mine give the result you expect?

senya2218:08:01

Let me try

eggsyntax18:08:28

Yeah, I was missing that inner "loop". I suspect you could do the whole thing fairly elegantly with a nested for, but I'd have to mess with it.

senya2218:08:18

@porglezomp:

(reduce graph-building-reducer graph ruleset)
CompilerException java.lang.RuntimeException: Unable to resolve symbol: graph in this context, compiling:(C:\Users\Simeon\AppData\Local\Temp\form-init1745806617618307544.clj:1:1) 

senya2218:08:44

@porglezomp: or should it be part of the task2 function?

porglezomp18:08:56

Oh, right, that should be dep/graph I think?

porglezomp18:08:29

The intention is that the symbol that I use to have as graph there should be the initial graph that you want to add to.

eggsyntax18:08:30

Probably (dep/graph) -- not the fn but the results of calling the fn.

senya2218:08:32

@eggsyntax: but somehow the graph seems right from its toString(). Weird...

porglezomp18:08:56

@senya22: if the result is the one @eggsyntax posted, it looks wrong from here, since it has things like sets of symbols depending on sets of sets, when you probably want single symbols depending on sets of symbols.

senya2218:08:15

@porglezomp: ah, yes - sets of sets seem wrong

senya2218:08:28

@porglezomp: are you able to run your snippet? It throws that exception for me...

porglezomp18:08:28

I haven’t set up a project with dependency, but I can do that if you’d like me to fiddle with it.

senya2218:08:48

I'd appreciate it

porglezomp18:08:57

But really, just change graph into whatever graph you want to start with, which might be (dep/graph) or something.

senya2218:08:34

change where?

porglezomp18:08:52

The graph in the reduce that’s not working for you. The initial value.

eggsyntax18:08:33

Yeah, should be (dep/graph) instead of graph.

senya2218:08:55

I tried that, still same effect

senya2218:08:01

let me retry

eggsyntax18:08:04

@senya22: want to post a snippet of the complete code as it stands? That'll help. I can also help w/ debugging it.

porglezomp18:08:07

I changed the snipped I posted.

senya2218:08:38

IllegalArgumentException No implementation of method: :depend of protocol: #'com.stuartsierra.dependency/DependencyGraphUpdate found for class: clojure.lang.Keyword clojure.core/-cache-protocol-fn (core_deftype.clj:554)

porglezomp18:08:41

Oh, I figured it out, I think

senya2218:08:48

OK, here's the snippet with @eggsyntax's suggestions incorporated:

porglezomp18:08:35

I fixed the snippet, it produces:

#com.stuartsierra.dependency.MapDependencyGraph{:dependencies {:c #{:b :a}, :f #{:e :c :d}, :m #{:e :k :d}, :k #{:t :p}}, :dependents {:a #{:c}, :b #{:c}, :c #{:f}, :d #{:m :f}, :e #{:m :f}, :k #{:m}, :p #{:k}, :t #{:k}}}

porglezomp18:08:46

Does the fixed one look like what you need? I had left the initial value of the reducing function as the wrong object.

senya2218:08:25

Yes, that eliminates the sets of sets

senya2218:08:52

Let me try to understand what how it's done now 🙂

senya2218:08:02

Thank you guys for help.

eggsyntax18:08:35

Happy to help! Nice work @porglezomp, esp if you don't have much clj experience yet 🙂

porglezomp18:08:51

I’ve done some lisp before, and I’ve used a lot of different programming languages, so picking up one more has gone pretty quick. @senya22 I can give a quick explanation of what my code’s doing.

senya2218:08:23

@porglezomp: please - greatly appreciated

porglezomp18:08:31

How familiar are you with reduce? You seemed to be getting it mixed up with map earlier, so how much detail about it should I give?

senya2218:08:08

Not too familiar

senya2218:08:52

I see you are starting with the initial graph as a starting accumulator to it

porglezomp18:08:15

The idea we’re working with here is that we need to apply dep/depend to the graph a lot of times, with different arguments. If we do (dep/depend somegraph :a :b) and then (dep/depend somegraph :a :c), it’s going to produce two completely different graphs, the second one doesn’t include the first.

porglezomp18:08:36

So we need to do (dep/depend (dep/depend somegraph :a :b) :a :c), and that will add both.

porglezomp18:08:35

(reduce f xs) will give us (f (f (f (f x1) x2) x3) x4) etc,

senya2218:08:59

does each (dep/depends) call return the same graph with more elements added?

porglezomp18:08:14

Yep, immutability at work.

porglezomp18:08:32

So we take reduce, and pass it an empty graph as its accumulator

porglezomp18:08:56

And then we walk over the list of rules, and apply one rule at a time, making the new graph the new value of the accumulator

porglezomp18:08:19

graph-building-reducer should really be called apply-rule.

porglezomp19:08:19

So I’ll refer to it as that. In apply-rule, we do a similar kind of reduce over the first element of [[:a :c :c] :-> :d]. We take some graph, and we apply (dep/depend graph :d :a), etc.

porglezomp19:08:47

So the call (reduce #(dep/depend %1 target %2) graph srcs) will produce something like (dep/depend (dep/depend (dep/depend graph :d :a) :d :b) :d :c).

leo.ribeiro21:08:52

hey guys, any example writting unit tests that uses mock data instead of the database? (it would be great if I can see how you test queries and insert/update methods)