My experiences are similar to yours, though less k8’s focused and more general DevSecOps.
it becomes a battle between custom-fitting and generalisation.
This is mentioned in the link as “Barely General Enough” I’m not sure i fully subscribe to that specific interpretation but the trade off between generalisation and specialisation is certainly a point of contention in all but the smallest dev houses (assuming they are not just cranking hard coded one-off solutions).
I dislike the yaml syntax, in the same way i dislike python, but it is pervasive in the industry at the moment so you work with that you have.
I don’t think yaml is the issue as much as the uncontrolled nature of the usage.
You’d have the same issue with any format as flexible to interpretation that was being created/edited by hand.
As in, if the yaml were generated and used automatically as part of a chain i don’t think it’d be an issue, but it is not nearly prescriptive enough to produce the high level kind of model definitions further up the requirements stack.
note: i’m not saying it couldn’t be done in yaml, i’m saying that it would be a massive effort to shoehorn what was needed into a structure that wasn’t designed for that kind of thing
Which then brings use back to the generalisation vs specialisation argument, do you create a hyper-specific dsl that allows you only to define things that will work within the boundaries of what you want, does that mean it can only work in those boundaries or do you introduce more general definitions and the complexity that comes with that.
Whether or not the solution is another layer of abstraction into a different format or something else entirely i’m not sure, but i am sure that raw yaml isn’t it.
Yes, I think yaml’s biggest strength is also its built-in flaw: its flexibility. Yaml as a data structure is built to be so open-ended that it can be no surprise when every component written in Go and using Yaml as a data structure builds their spec in a slightly different way, even when performing the exact same functions.
That’s why I yearned for something like CUE and was elated to discover it. CUE provides the control that yaml by its very nature cannot enforce. I can create CUE that defines the yaml structure in general so anything my system builds is valid yaml. And I can create a constraint which builds off of that and defines the structure of a valid kubernetes manifest. Then, when I go to define the CUE that builds up a KubeVela app I can base its constraints on those k8s constraints and add only KubeVela-specific rules.
Then I have modules of other components that could be defined as KubeVela Applications on the cluster but I define their constraints agnostically and merge the constraint sets together to create the final yaml in proper KubeVela Application format. And if the component needs to talk to another component, I standardize the syntax of the shared function and then link that function up to whatever tool is currently in use for that purpose.
I think it’s a good point that overgeneralization can and does occur and my “one size fits all” approach might not actually fit all. But I’m hoping that if I finish this tool and shop it to a place that thinks it’s overkill, I can just have them tell me which parts they want generalized and define a function to export a subset of my CUE for their needs. And in that scenario, I would flip and become a big proponent of “Just General Enough”. Because then, they can have the streamlined fit-for-purpose system they desire and I can have the satisfaction of not having to do the same work over and over again.
But the my fear about going down that road is that it might be less of an export of a subset of code and more of building yet another system that can MAD-style generate my whole CUE system for whatever level of generalization I want. As you say, it just becomes another abstraction layer. Can’t say I’m quite ready to go that far 😅
My experiences are similar to yours, though less k8’s focused and more general DevSecOps.
This is mentioned in the link as “Barely General Enough” I’m not sure i fully subscribe to that specific interpretation but the trade off between generalisation and specialisation is certainly a point of contention in all but the smallest dev houses (assuming they are not just cranking hard coded one-off solutions).
I dislike the yaml syntax, in the same way i dislike python, but it is pervasive in the industry at the moment so you work with that you have.
I don’t think yaml is the issue as much as the uncontrolled nature of the usage.
You’d have the same issue with any format as flexible to interpretation that was being created/edited by hand.
As in, if the yaml were generated and used automatically as part of a chain i don’t think it’d be an issue, but it is not nearly prescriptive enough to produce the high level kind of model definitions further up the requirements stack.
Which then brings use back to the generalisation vs specialisation argument, do you create a hyper-specific dsl that allows you only to define things that will work within the boundaries of what you want, does that mean it can only work in those boundaries or do you introduce more general definitions and the complexity that comes with that.
Whether or not the solution is another layer of abstraction into a different format or something else entirely i’m not sure, but i am sure that raw yaml isn’t it.
Yes, I think yaml’s biggest strength is also its built-in flaw: its flexibility. Yaml as a data structure is built to be so open-ended that it can be no surprise when every component written in Go and using Yaml as a data structure builds their spec in a slightly different way, even when performing the exact same functions.
That’s why I yearned for something like CUE and was elated to discover it. CUE provides the control that yaml by its very nature cannot enforce. I can create CUE that defines the yaml structure in general so anything my system builds is valid yaml. And I can create a constraint which builds off of that and defines the structure of a valid kubernetes manifest. Then, when I go to define the CUE that builds up a KubeVela app I can base its constraints on those k8s constraints and add only KubeVela-specific rules.
Then I have modules of other components that could be defined as KubeVela Applications on the cluster but I define their constraints agnostically and merge the constraint sets together to create the final yaml in proper KubeVela Application format. And if the component needs to talk to another component, I standardize the syntax of the shared function and then link that function up to whatever tool is currently in use for that purpose.
I think it’s a good point that overgeneralization can and does occur and my “one size fits all” approach might not actually fit all. But I’m hoping that if I finish this tool and shop it to a place that thinks it’s overkill, I can just have them tell me which parts they want generalized and define a function to export a subset of my CUE for their needs. And in that scenario, I would flip and become a big proponent of “Just General Enough”. Because then, they can have the streamlined fit-for-purpose system they desire and I can have the satisfaction of not having to do the same work over and over again.
But the my fear about going down that road is that it might be less of an export of a subset of code and more of building yet another system that can MAD-style generate my whole CUE system for whatever level of generalization I want. As you say, it just becomes another abstraction layer. Can’t say I’m quite ready to go that far 😅