Fork me on GitHub
#beginners
<
2021-04-27
>
piyer04:04:41

I am trying to figureout how to use honeysql DSL with migratus. https://github.com/yogthos/migratus#modify-sql-fn has an option to format the SQL but I have to write edn style dsl and name that as *.sql.

seancorfield04:04:53

@munichlinux You're beyond the edge of what these libraries are intended to provide -- but it's an interesting idea. In general, I don't find a DSL worthwhile for DDL for SQL migrations but I'm interested to hear what value you are getting from it?

piyer16:04:26

@U04V70XH6 Thank you. 1. A level of abstraction over the database (less important, in real use I never had to swap the database). 2. I kinda like the brevity of DDL that makes it really simple rather than writing a raw SQL.. I use to write nodejs for web. I used knex , here is an example:

const tableName = 'session';

exports.up = (knex) =>
  knex.schema.createTable(tableName, (table) => {
    table.string('sid').primary().notNullable();
    table.json('sess').notNullable();
    table.timestamp('expire').notNullable();
    table.timestamp('created_at').defaultTo(knex.fn.now());
    table.timestamp('updated_at').defaultTo(knex.fn.now());
  });

exports.down = (knex) => knex.schema.dropTable(tableName);

seancorfield17:04:08

Hmm, I find plain SQL easier to read but I can see how, with a few helper functions for common columns or column types, you could use more concise code to describe tables… and migratus does let you use code-based migrations although the format is a bit verbose. Maybe @U050CBXUZ would be open to providing a more concise version that leveraged HoneySQL to format the result of calling DDL-generating functions? Not sure what that would look like tho’… just spit-balling an idea…

yogthos17:04:23

I'd be open to making it easier using HoneySQL, it shouldn't be too hard to add a layer that could parse HoneySQL markup and generate SQL strings that would then be consumed. Could leverage existing edn migrations. Currently, there are :up-fn and :down-fn keys, so perhaps could have :up-honey and :down-honey that would point to HoneySQL markup.

seancorfield17:04:53

The “benefits” would likely come from having helper functions that generated common DDL fragments so it would need to be actual code — similar to how the EDN/code migrations work today — except that the functions would produce the ["SQL"] vector that migratus would then run honey.sql/format on.

seancorfield17:04:53

Allowing :up-fn / :up-honey / etc to be a fully-qualified symbol so the :ns could be omitted might also be nice.

seancorfield17:04:53

And there would need to be some way to specify options for the honey.sql/format call I guess, either globally, or on a per-migration basis.

seancorfield17:04:03

I mean, maybe there’s also an avenue for providing HoneySQL DSL as EDN but I don’t think that’s as valuable as having functions to generate the ["SQL"] vector — the DSL is pretty verbose for DDL stuff, especially table/column specs.

piyer19:04:21

@U050CBXUZ IMO, Here are my two options. 1. Let people write a clj file for migration. It will be foo-123.up.clj foo-123.down.clj 2. you already have a way to run a preprocessor, I can say the default preprocessor is SQL or honeysql or tomorrow's new thing. I personally like 1. As long as I have a public method up that returns string, I can do whatever I want.

piyer19:04:54

what do you guys thing?

seancorfield19:04:11

@munichlinux The .edn version is effectively 1. It’s declarative and based on a specific function for up and another for down.

seancorfield19:04:00

How would an arbitrary .clj file even be used? The namespace should match the filename for a start so you can’t have . in that part.

yogthos19:04:27

right, the current .edn version is seems very similar to option 1, it sounds like the biggest win would be in having a DSL for describing DDL fragments

piyer19:04:39

@U04V70XH6 since these files are generated. we can generate migrations.123 as the ns.

piyer19:04:45

I am trying to keep this independent of edn or honeysql.

piyer19:04:23

(-> (all-ns)
     (filter (starts with migrations)
      (sort by timestamp)
      (doseq [ns all-ns]
          (ns/up)))

seancorfield19:04:29

@munichlinux The migrations need to follow a specific naming patterns so they can be run in order and all checked off in the tracking table. (right @U050CBXUZ)

seancorfield19:04:52

Oh, are you proposing not even using migratus and writing your own custom migrations stuff?

piyer19:04:12

nope, I proposing to use migratus . I was just outlining how the clj will work.

piyer19:04:32

ATM, migratus can generate the file names. I propose to generate the file with an ns if we have to take the clj approach.

yogthos19:04:37

yeah, migratus needs to have a numeric pattern in the migration name to track the order of migrations

yogthos19:04:12

it might also make sense to make a separate library that leverages migratus for this

Aron05:04:47

I am getting a

Encountered error when macroexpanding cljs.core.async/go.
Encountered error when macroexpanding cljs.core/loop.
Syntax error macroexpanding.
And I have no idea what to do about it, it should work as far as I understand it. I tried to wrap it in macroexpand which expanded it but where to go from there? It looks fine.

alexmiller05:04:22

I don't know what this might be, but generally it's helpful to include the code reproducing the problem

Aron05:04:17

I am trying to recreate the minimal example.

Aron05:04:05

I think I got it, I had <! in my recur, I now put that inside the loop with a let binding and it doesn't say it's bad

Aron05:04:44

Why is it that I only get these ideas after publicly asking for help? I tried to figure out for hours alone.

andy.fingerhut05:04:43

Rubber ducking

☝️ 12
😂 3
sova-soars-the-sora18:04:44

same here! i will often type out a question to the slack or to stack overflow and just typing it out to explain it often gives me more insight into the issue. First I hear of "rubber ducking" now I might have to get a comically large one for the desk...

Aron05:04:13

reasonable guess, I guess 🙂 although this particular time it felt different. I already removed most of the code before writing and I just randomly figured maybe it's the <! in recur

hiredman05:04:47

A <! In a recur should in theory work fine, but the go macro on the cljs side of core.async isn't as a robust, and sometimes the way cljs macroexpands code makes it almost impossible for the go macro to rewrite it (some macros expand to a blob of js which the go macro can't rewrite)

hiredman05:04:35

So you may have hit a bug in core.async, an annoying tricky bit in how core.async and clojurescript interact, or actually done something incorrectly yourself

Aron05:04:08

If it happens lots I would guess other than the last and provide actual examples, right now I am just extremely happy I can move on. Need to finish this stuff and start other things because ... well, life.

Pablo19:04:03

Hello everyone, I have a doubt. Can specs be defined such that only their generators are included in the development environment? not in production

seancorfield19:04:36

@pablog Do you mean custom generators? Those are provided as functions, so they are only called if you actually do something generative to a Spec. And you can write functions such that they only load their dependencies if they are called so you can omit those dependencies from a production build if you want.

seancorfield19:04:57

We have generators that depend on test.check which is only available at dev/test time for us.

Pablo19:04:41

Yes, custom generators. For example, if I define something like:

(s/def ::name
  (s/with-gen string?
    #(clojure.test.check.generators/elements #{"John" "Marie"})))
And in my application I use :name in order to validate some data, I don’t want that clojure.test.check.generators stuff gets loaded on production. Even compiled and packed.

Pablo19:04:20

The problem is that spec.alpha uses an older version of test.check

seancorfield19:04:31

(s/def ::name
  (s/with-gen string?
    #((requiring-resolve 'clojure.test.check.generators/elements) #{"John" "Marie"})))

noisesmith19:04:44

isn't it the case that you can use clojure.spec.gen.alpha safely, and if you call it it auto-loads test.check?

noisesmith19:04:10

IOW it errors if used and check.test isn't provided, but otherwise compiles without complaint

seancorfield19:04:13

Right, but @pablog wants a more up-to-date test.check

noisesmith19:04:29

then depend on the version you want?

seancorfield19:04:31

(which it would use if it was specified as a test dep, right?)

alexmiller19:04:44

You can specify a dep on a newer version if you like

alexmiller19:04:40

spec just (dyna)loads what's on the classpath

seancorfield19:04:22

Yeah, the key thing is not referring to clojure.test.check statically in your code. Use requiring-resolve and it won’t try to load/compile it unless that code is evaluated.

Pablo19:04:49

Sorry, I think that I missed some important fact. How can I use test.check/let ?

Pablo19:04:42

dynamically

alexmiller19:04:03

you can't do that with spec

noisesmith19:04:04

the simplest thing is to only use it in code that is run locally, and just require test.check

noisesmith19:04:31

the file referencing test.check can be kept out of your jar entirely

Pablo19:04:57

Ok, thanks a lot! 🙂

hiredman20:04:43

test check's let is a macro that expands from something like (check/let [a b] c) to something like (check/bind b (fn [a] c)) and bind is a function, so using it dynamically is a lot easier than a macro

3
hiredman20:04:22

(test.check generators are a monad, and check/let is a monad comprehension for it)