This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2024-05-07
Channels
- # announcements (16)
- # asami (15)
- # babashka (12)
- # beginners (38)
- # calva (32)
- # cider (1)
- # clj-commons (9)
- # clj-otel (4)
- # clojure (57)
- # clojure-europe (43)
- # clojure-korea (1)
- # clojure-nl (1)
- # clojure-norway (13)
- # clojure-uk (4)
- # clojuredesign-podcast (9)
- # clojurescript (10)
- # cursive (5)
- # datahike (9)
- # deps-new (2)
- # events (1)
- # fulcro (8)
- # hyperfiddle (7)
- # kaocha (1)
- # lsp (2)
- # malli (3)
- # nrepl (2)
- # off-topic (19)
- # releases (3)
- # ring (10)
- # shadow-cljs (4)
- # sql (14)
- # xtdb (57)
- # yamlscript (2)
anyone using dbt for db model management in production? Any helpful tips, tricks, tools and experiences to share?
Not helpful if you already use dbt, but in case you're just deciding on it, I myself would probably steer clear of it: • https://medium.com/@laurengreerbalik/dbt-clouds-new-pricing-model-the-sinister-phase-ii-fe845aee21d9 • https://medium.com/@laurengreerbalik/dont-do-accounting-in-dbt-cloud-489eb5613dd1
Not sure if i can influence decision. We are not using it but there is big push for it for some reason. I know nothing about it so i was asking around. Will check these articles. Thanks! dbt was only mentioned to me in the context of 'creating tables' but i am not sure of any other detail
Of course, I have no idea about the specifics of your situation, but on the surface it sounds like some manager attempts to enforce a technical decision. If so, the big push can easily be coming due to the stellar work of the sales team from the dbt's side. :) This crap happens all the bloody time in IT, and all it does is siphon money and time from your company and into the pockets of the ones offering "solutions".
dbt is really nice in the hands of data engineers/scientists and helps facilitate a LOT of good habits in an ecosystem that's kinda the wild wild west
i'd not use it on the software side or for "model management", there are better migration tools for that sort of thing
@U02EA2T7FEH are there alternatives that you'd use?
it really depends on what they mean by "creating tables" and "model management" so it's hard to say
https://github.com/yogthos/migratus has been my go-to migration tooling
but, again, if they meant "i want a table that i can manage in my db that only has data from trial accounts" or something like that
dbt would fit pretty well
> i want a table that i can manage in my db that only has data from trial accounts Wouldn't a plain view be sufficient for that?
probably; though dbt can materialize the queries as views, not just tables. but who is managing that view? if it's not a view, how often do they need the new data added? how technical are the users? can they write sql queries? if so, as a data engineer, not writing queries for people sounds really attractive
the data world is in desperate need of a lot of things: • reusable code (notebook are a blight and are almost used exclusively from my experience) • local development (notebooks strike again...) • proper testing (guess who?) dbt isn't a magic bullet. it won't be much use (without paying a premium) if you have a LOT of data and you have a short SLA. but it does solve those 3 problems pretty well and is generally a good fit if you: • don't have a ton of data • have data scientists that need exploration and flexibility • want to provide some kind of self-service interface to your raw (or slightly curated) data sets