@bcbradley: The problem with that approach is that it requires you to load your working set into memory. There's not a lot that can't be done fairly straightforwardly in SQL, especially once you learn the more advanced features like window functions, group by, and having clauses.

Two questions: 1) If I'm walking a recursive structure and producing a nested sequence of strings as a result ("a" ("b" "c" ("d"))), is there something more efficient than a single flatten pass after the walk to flatten it? 2) If certain items in the recursive structure need to be watched for the first and last of their type within one type of node, and between another type of node, how would you best handle that?

(The first and last need extra tags in the resulting strings)

depending on how you are walking the tree, you should generate a flattened list as you walk it

easiest for depth first, but possible for breadth too (minikanren and core.logic use an interleaving mechanism to do it)

flatten is very gross, its use signifies sloppiness to me

are clojure's futures and promises using java8's native libraries for those ?

java.util.concurrent.Future is older than Java8 and seems to be in use if you check the source e.g. here https://github.com/clojure/clojure/blob/clojure-1.8.0/src/clj/clojure/core.clj#L6697 (updated link to 1.8)

funcool/promesa works with java8 completable futures

thanks @niwinz !

@seancorfield: Yesql doesn’t support dynamic query generation. It’s likely it won’t. I’ve never used something like that that doesn’t become a real mess. If that’s what you need, you’re better off either pushing that complexity back into the database (with views, etc.), or pulling it forward into the language and accepting that you’ve given up the simplicity of SQL for the sophistication of a programming language. That’s a long way of saying, I don’t think it works very well*, if that’s what you need you should look for a different route. (* Esqueleto is the only “dynamically compose SQL queries” approach that I know of, that I’d rate.)

i actually like Suricatta as well

it's great for composable sql queries

I personally think there definitely is a case to be made for composable SQL queries

I thought hugsql was pretty composable?

Don't know about HugSQL, since you have to hand code each query, but HoneySQL is pretty nice for composing queries

full circle, honeysql was mentioned already (scroll up) 🙂

I like honeysql; I don’t want a dsl to compose queries, I want a data structure that can be transformed into a query.

@hiredman: Thanks.

The recursive structure I'm walking shows lists as a succession of :ordered-list or :unordered-list items. I need to convert each of those to an <li>, but I also need to wrap each whole list in <ol> or <ul> tags. I figure I can split lists when I descend into a block, and possibly on a paragraph. Would you pre-process the data structure to flag starts and ends of lists, and then go back through to process the HTML? Or is there some more elegant way I'm not thinking of?

@seylerius: sounds like you'd want to split the lists with partition-by as a first step (they're flat initially, right?)

I mean the split the items

@seylerius: have an example of the input/output you're looking for? likely very easy to do with specter

I wrote a cute little BBCode parser in Clojure today.

Falls flat on its nose right now when it faces invalid input, though, so that needs a slight bit of polishing.

@stathissideris: Nope, not flat to start. Sequence of maps, some of which contain sequences of maps (etc.)

@nathanmarz, @stathissideris: Here's a sample. Source file: http://sprunge.us/PQBX Parsed data structure: http://sprunge.us/AXaF

is this org mode? 🙂

oh it is (just read the first line)

so are you asking whether to nest level 2 subsections under level 1?

I would

it makes more sense but it also has certain advantages (for example, you can get all the contents for a level 1 section in one go)

an org mode parser would be a great thing to have

@seylerius: yea super easy with specter

(use 'com.rpl.specter)
(use 'com.rpl.specter.macros)

(transform
  [ALL
   :content
   (multi-path
    (continuous-subseqs #(= (:line-type %) :ordered-list))
    (continuous-subseqs #(= (:line-type %) :unordered-list)))
    ]
  (fn [aseq]
    (let [type (-> aseq first :line-type)]
      [{:line-type (if (= type :ordered-list) :ol :ul)
        :content (setval [ALL :line-type] :li aseq)
        }]
      ))
  data
  )

here's a gist showing the before/after on part of what you posted: https://gist.github.com/nathanmarz/1a5728e8a0e12e08c02d98aa1df675aa

I didn't see the recursion in the example you gave, but this can easily be made recursive as well

@nathanmarz: "The First Section" is missing from your gist

for brevity, I only took part of the sample input

oh ok

@stathissideris: More specifically, I'm trying to capture starts and ends of lists so I can wrap them in <ul> and <ol> tags as needed.

I'm expanding upon @gmorpheme's organum. The basic version of organum (with a bugfix I added because it was parsing table-row as #".*" due to excess escapes) can be found at https://github.com/seylerius/organum

Going to fold my parsing and formatting additions into organum, probably.

@seylerius: I would solve this with reduce. The step function passed to reduce would act as a state machine and achieve what you want.

reduce is basically a “wrapper” around loop/recur and will give you a lot of control and ease in this case

if I wasn’t swamped at work I would make an example but I can’t, I have this old blog post though. The bottom of the post is what I’m talking about http://jcswart.github.io/2014/01/25/clojure-like-im-five-reduce-functions/

notice how I keep “state”

you can do the same to detect start and ends of the lists

@jswart: Thanks. I'll need to take a look at that. I'm still learning about reduce.

@jswart: I see... You're passing state inside a vector containing your accumulation. Could probably toss that afterwards by having the very first call be something like (first (reduce dispatch-node org-nodes)), if dispatch-node returns [parsed-nodes state]

yep, however you want to set up

could be a map, vector, etc.

btw, these days it's recommended to use the init argument to reduce (reduce f init collection)

rich has mentioned the implicit first item of collection as init was a bad idea

"bad" is maybe too strong - perhaps "unnecessarily complex" :)

What would the better design have been in retrospect? Implicit init of (f) when it's not provided?

explicit init always

aka the IReduceInit interface

yea, that makes sense

is the reason for having implicit init with transduce just for consistency with reduce?

yeah

prefer explicit over implicit (I remember that tenet from my python days)

Hi. I’m trying to use lein uberjar and I’ve got a skip-aot (some dependencies look for env vars and so on). How do I make this executable?

once you have the jar you can run java -jar YOUR_JAR, is that what you mean?

well so I am trying that

I am working from this example: https://gist.github.com/gja/efa1a7beea9ecb52e906

but running with java -jar, I can’t find my core namespace

Ah, sorry. There might be an :aot you have to set as well but I don’t use lein so I’m not really sure.

If you're trying to mine useful information out of the output of something like ffmpeg that modifies its output stream, and you want to use that info for a progress bar, how would you go about doing that?

grep

...

Your question is kind of vague, but that is what I would do.

@jswart Specifically what I'm aiming at is the total length of one of the input files (which doesn't get edited), and the current time through the encoding process (which changes as it progresses, eventually reaching the total length).

The standard output stream doesn't share that usefully.

does it share anything? you could grep for the pieces, put them in a file and the wc -l the file. The number of lines == the progress.

Doing this back and forth between clojure doesn’t seem super fun though, and if you have a lot of files its like just better to show which files are done and which are pending, etc.

But I have no idea your use case

@seylerius I'll happily incorporate any contributions to organum. alternatively happy to redirect people to your fork if you prefer as realistically I have zero time to spend on it!

@gmorpheme: Glad to finally catch up with you. I'm probably going to be doing a fair bit to it, so perhaps switching to my fork will work best for the longer term. Planning to fill out the parsing some (inline formatting, table items in rows, etc.), and add basic HTML output (perhaps in hiccup format).

Need both of those for for an editor I'm making in support of another project.

(The bigger project is a decentralized office suite focused on activists, analysts, and journalists. Main tools: org-editor; distributed, encrypted file storage; evidence storage; knowledge graph database; team-oriented comms suite)

@jswart: Here's some sample output: http://sprunge.us/bfIa The duration of the second of the two files is 5.4s. The first input file is a picture. It's a simple program to slap a GUI on the ffmpeg command that glues an image to an audio clip.

@seylerius: sounds great! I'll keep an eye on it. I'll change the readme to point people at your fork when I get a chance

@gmorpheme: I'll mod the readme and namespace on my end, then, to differentiate, and to credit you for creating the original project.

@seylerius Thanks. Most appreciated. Probably tomorrow now (pretty late here).

@gmorpheme: Thanks for creating it in the first place. It's particularly handy. Also, if you find yourself with stupid amounts of free time after I've got the basic structure ready, this project will be huge and in need of good coders.

Oh I'm not short of stuff to be working on I promise you:) But I'll watch with interest at least