Fork me on GitHub
#off-topic
<
2020-02-07
>
Alex Fedotov09:02:47

Hey there! ✌️ My name is Alex, I’m a product designer at Miro (http://miro.com) — the online collaborative whiteboarding platform for distributed teams. We want to make the 1st session in our product better, and if you haven’t used Miro before, I’d like to invite you to participate in a quick usability testing. If you’re open to taking part in my small research, please go through invitation form ☺️ https://miro-survey.typeform.com/to/TlRH2r

👏 4
👍 4
eccentric J17:02:01

Does Miro use Clojure? I use Miro fairly regularly and I am curious about what it was made with.

jjttjj18:02:43

I’ve been stuck on this problem for a bit and i wonder if anyone knows of any other work related to what I’m trying to do or if I’m on a totally wrong track and there’s any other things I could look at, or have any other input that might jog this “writers block” I have time series data on a (clj) server. A time series is a column map (ie columns to sequences). I want to access it from a cljs client. I basically just need to specify a series and a time interval i want the data for.  The tricky part is I also want to be able to run calcuations on these series, computed on the server but which are specified on the client. Calculations can be run both on series and other calculations made in the same request.  I’ve been messing a lot with mini data oriented query langs for this, something that might look like:

(q
 '[:let [msft [:seek [{:symbol "MSFT" :res "1 day"} range1]]
         aapl [:seek [{:symbol "AAPL" :res "1 day"} range1]]
         relative [:calc [:divide (:close msft) (:close aapl)]]
         rel-ema [:calc [:ema 20 relative]]]
   ;;return 
   [msft aapl rel-ema]])
But I’m starting to wonder if it might be better to just send some arbitrary code to the server to be executed in a sandbox, or with whitelisted functions
`(let [msft     (my.ns/seek [{:symbol "MSFT" :res "1 day"} range1])
       aapl     (my.ns/seek [{:symbol "AAPL" :res "1 day"} range1])
       relative (my.ns/divide (:close msft) (:close aapl))
       rel-ema  (my.ns/ema 20 relative)]
   [msft aapl rel-ema])
Where all the my.ns functions above are executed on a server. Has anyone ever need to do this sort of remote querying/calcuation combination?

vemv18:02:51

what's the primary concern driving you to want specifying computations client-side, but executing them server-side?

jjttjj18:02:13

It's for a cljs user interface which will show a chart and allow users to add and configure calculations. But some of the calculations could be cpu intensive

👍 4
vemv18:02:10

Got it. Can't help much I'm afraid... what are your thoughts on datalog? There are many flavors in clj iirc

jjttjj19:02:09

I do like datalog, but the logic constraints are not particularly useful because the series lookups will be basically just a key lookup and then restricting the value to a time range. I guess this is possible with predicate in datalog. I suppose I have to look into more how custom calculations might be run on datalog results

👀 4
hiredman18:02:04

I might start with something like honeysql's dsl, which is to some degree "sql as clojure datastructures" and use that as my query language and write an interpreter for it on the server side

jjttjj19:02:39

Cool yeah I've used honeysql a tiny bit but didn't think about it for this context, that's an interesting starting point though thanks.

jjttjj19:02:40

I guess the main question is is how to handle things like runing-average type calculations (and not just strict aggregations) and naming intermediate calculations that are used for other calculations (my sql is a bit rusty and I'm not sure if these things are supported out of the box)

hiredman18:02:35

(because it sounds like you want sql, but who wants to parse sql?)

lilactown19:02:03

before writing an interpreter or evaling code I would see if I could just do a more static, purpose-built query DSL:

;; query body
'{:ops [{:op :seek :ref msft :seek/symbol "MSFT" :seek/res "1 day" :seek/range range1}
        {:op :seek :ref aapl :seek/symbol "AAPL" :seek/res "1 day" :seek/range range1]}
        {:op :divide :ref relative :divide/numerator (:close msft) :divide/denominator (:close aapl)}
        {:op :ema :ref rel-ema :ema/points 20 :ema/value relative}]
  :results [{:ref mstf} {:ref aapl} {:ref rel-ema}]}
(I guessed at what some of the args meant) this way you can essentially implement handling each :op as a multimethod dispatch, keeping the state of the query in a context that gets passed to each operation it will also be much easier to create programmatically by the UI since it is more structured than a general purpose query DSL

lilactown19:02:31

data DSLs like the ones above are much easier for humans to read/write, but harder to construct. unless your UI is expecting the user to literally write these queries, I think having a more AST-like DSL like above is better. It will be easier to generate, easier to validate, etc. If later you want to add the more human-readable version, you can parse the code into the AST format you’ve already created

jjttjj19:02:07

That seems very promising, thanks! and yeah you guessed correctly on the arg meanings 🙂

jjttjj19:02:25

When you say AST, the "tree" in question is derived from the "dependency tree" of the refs right?

jjttjj19:02:15

or just the :op+ :results map

lilactown19:02:51

I was referring to the :op + :results map as similar to an AST

jjttjj19:02:59

gotcha, great

lilactown19:02:05

the example I gave does not very “tree-like” in this case because your example queries were very flat. but if you had nested operations, it would look very similar to an AST

jjttjj19:02:22

got it i think, so there might be nested op-maps so it could look more like

'{:ops [{:op               :divide :ref relative
         :divide/numerator {:op          :seek  :ref      msft
                            :seek/symbol "MSFT" :seek/res "1 day"
                            :seek/range  range1}
         
         :divide/denominator {:op          :seek  :ref      aapl
                              :seek/symbol "AAPL" :seek/res "1 day"
                              :seek/range  range1}}
        {:op :ema :ref rel-ema :ema/points 20 :ema/value relative}]
  :results [{:ref mstf} {:ref aapl} {:ref rel-ema}]}
And internal maps could just always be interpreted as op-maps? (I know my problem is still pretty vague and there's no definitive answer here because it depends what I'm going for)

lilactown19:02:47

right, you could do something like that. you’d need to ensure that :ref symbols in nested operations don’t conflict with :ref symbols higher up

lilactown19:02:16

and I’d probably do something more like:

:divide/numerator {:ops [{:op seek :ref msft ...}] :results {:ref msft}}
so that you could re-use your interpreter recursively. but this is getting quite a bit more complex; I would try to flatten it on the client side first

👍 4