@hobosarefriends The landscape is quite different these days, compared to when I was moving up into that role. I'd say you will need a good grasp of cloud-based services and understand their trade offs -- which wasn't really a thing twenty years ago. You need to be able to recognize engineers who can do the practical research and evaluation of new technologies that you want to consider introducing (you can't do it all yourself and, besides, you need buy-in from engineers if you're planning large scale changes to the systems they work on).

You also need to make a fundamental decision about which architect role you want to go for: software architect and systems architect are quite different, for example, and there are several other specialized architect roles (more these days than there were twenty years ago). I chose software architect but also had to dance on the edge of systems architecture (I was Macromedia's senior architect for all IT projects, and I had a team of architects that covered web infrastructure, database, and information systems -- we didn't have a dedicated systems architecture function back then).

At Macromedia, being the senior architect meant C-level meetings so good communication skills with the heads of various divisions was key, especially with non-IT-savvy executives. It also meant good negotiation skills since you often need to get buy-in from management across all the departments you're trying to affect.

That makes sense! I suppose trying to deploy apps in different providers and in different ways would be a good investment of my time then. I’ve been leaning more to software architect, which I suppose would probably involve more code-style and library choice decisions. I suppose working as a project lead would probably have benefitted me more in this regard, and I will express my interest in that in the future. For now, I suppose continuing to freelance will also help in most of these cases.

Thanks for taking the time to answer! I appreciate the level of detail and the concrete examples.

Re: leadership -- yeah, the choices are either "team lead" -> "team manager" -> moving up the "people" side of things vs "technical lead" -> "technical analyst" -> "architect" -> moving up the "technical" side of things. But a lot of companies don't really have a distinct, recognizable "architect" role.

For example, when Adobe acquired Macromedia, my role was essentially eliminated -- I was "Director" level at that point, but had no direct reports since I floated from large project to large project advising on software choices and, well, architecture. Adobe told me "We do not have architects" so I got "demoted" to a glorified technical lead at a "Manager" level... which turned out to be so frustrating that I quit within a year. Macromedia was a relatively easy company in which to effect change -- Adobe was very set in its ways and resisted nearly all my efforts to change how stuff was done (after several initiatives I tried to steer through IT had gotten "stuck" due to folks not wanting to change anything, I gave up and focused on "team management" instead, which I hated really).

Adobe does have a reputation for resisting change, even on their product side… My limited experience with large companies has been as a consultant, which is probably different from working with them directly. Since these more “Bigger picture” type roles are much more prevalent in larger companies, I should probably weigh my preferences with the downsides of working for these types of companies (including getting acquired). Maybe I’ll do what my father did and go the entrepreneurial route. Though I can’t say I’m very fond of the risks involved in that path 😅

Yeah, I've been a consultant in both the UK and the US, working with clients worldwide. It has a lot of pros, but also a lot of cons. I don't know that I would go back to it at this point -- the variability of income and sometimes the high-stress crunch of being driven by the client's needs can be very wearying 😐

Re .git - because it can be HUGE. And it should not be needed for your app. Re GIT_SHA - because you can deploy by multiple means, even the very same app on the very same dynos (talking about Heroku here), some of which don't know anything about Git.

Sounds like you just need to implement some versioning, maybe the one that uses sha.

I have a plugin that adds the git status info to your resources, so that it's present for examination in an artifact

a few others have done similar

@U2FRKM4TW - .git can be huge: there is a lot of "tweaks" to "clone just the current commit" - don't know anything of git: yeah, but the thing that "cloned the source" know. It can say to CI pipeline "i successfully download the current source, place include COMMIT_ID=123 in the pipeline env". Even if the trigger was launched by one push, and it clone another another (more recent) commit. And just include a extra info @U051SS2EU Do you commit a UUID/sha into resource? I also do it in another project. but it generate a lot of useless commits I can't understand how both "how can it be hard to heroku" either "how can it not be included on platform basic requirements"

my understanding is that outside pathological cases / intentional deception, a git hash is unique within a repo, and when applied to that repo exposes the exact state of every file

so all you need is the current git hash, plus verification that there are no unstaged changes (or an enumeration of those changes in the worst case)

that plus access to the repo exposes all the info

there's no need to commit the file exposing the sha into git - in fact that seems quite silly

Also, heroku shows Deployed 3517b1fc in dashboard So this information is present in the pipeline, it just not exposed in "build"/"run" trigger

I don't use heroku myself, but according to https://devcenter.heroku.com/articles/buildpack-api#bin-compile-summary the commit hash should be available as SOURCE_VERSION during the build. For CodeBuild you can use CODEBUILD_RESOLVED_SOURCE_VERSION

if you need it somewhere else in CodePipeline the CodeCommit and Github actions ouput a CommitId variable that you can access by adding a namespace to the Github/CodeCommit action and referencing as #{WhateverNamespace.CommitId} in some other action.

@U015879P2F8 I'm also heroku noob ATM But "buildpack" is a "create your own environment" option ATM I'm using the "docker buildpack", provided by heroku itself But thanks, I will try to see in github if I PR their "docker buildpack"

There is no "docker buildpack" docker is just "a magic env" inside heroku

are you sure the sha is not present in CI? try performing env | grep CI_ the CI_ prefix is quite conventional across providers, you might find a wealth of info there

already printed all env on heroku on AWS I don't interact directly, it's de "ops" team but I'm waiting the COMMIT_ID for more then a month to add at audit info

does heroku build the container images or do you build/publish them in aws and just deploy from a registry on heroku?

No channel that I'm aware of, but our Discourse is a great place to ask questions: https://discourse.metabase.com/

I've worked with twitter APIs doing graph analysis. Your best bet is to do an estimate on the amount of data you need to process and make some stress tests. Ideally tests where your code isn't all on one machine so you see the real impact of network overhead. Either of those DBs might work, or neither. We found kafka always held up under even the biggest loads we could throw at it, but it's not a graph db or even a db, you need to walk through the topic afterward to construct a view.

thx for the tip