To start, I think your comparison of Half-Dogai to Half-Molydeus is very well thought out and you bring up some great points. It definitely seems to me, from your analysis that the subrace needs some work. That being said, Banshrae seems like it's a solid subrace, so it might be helpful to compare the two side-by-side. Here we go:
"Half-Dogai" (secret eXtremely Rare)
- FCs: Fighter, Barbarian / BC: Assassin
- STR +8, DEX -2, CON +2, INT +4, Hide +8, Move Silently +8
- Free Feats: WP: Exotic, Armor proficiency Heavy/Medium, ESF: Discipline/Parry, Cleave, Great Cleave, Bullheaded
- Other: +1 die to Caustic Weapon
"Banshrae" (secret eXtremely Rare)
- FCs: Monk, Rogue / BC: Assassin
- DEX +8, INT +4, Hide +8, Move Silently +8
- Free Feats: Great Dexterity I/II, WF/GWF: Assassin Dagger, Snakeblooded
- Other: Very Fast
So the first thing that sticks out to me is the difference in total ability score bonuses/penalties when you include those from the base races. Half-Dogai gets +12 from the subrace and a total of -2 from Half-Orc. Banshrae gets +12 from the subrace and a total of +/-0 from Halfling. Already thus far into analyzing the two seems that Half-Dogai might be better served by a total +14 to stats from subrace. I think it might be a bit much to give them +8 STR and +6 INT, as I don't believe any other subrace gets two large bonuses like that, but it might be helpful to either simply remove the -2 penalty to DEX, or move it to WIS and give a +2 bonus to DEX to help with AC and DEX checks.
In regards to feats, it's actually a bit more complicated since Half-Dogai gets a total of 8 and Banshrae only gets a total of 5. That being said, it's important to also look at the quality of the feats that are given. Firstly, I feel like WP: Exotic is far from optimal because it encourages STR assassins to invest in an exotic weapon when they may want something else, be it Martial (where there are plenty of good options) or even Simple, like Morningstar. I definitely agree that the armor proficiencies seem a bit redundant, as I assume that most, if not all, STR assassins are going to be taking at least one, most likely 2 Full BAB classes pre-epic which will already grand those feats, as you've pointed out. ESF: Discipline/Parry are nice, but there are far better options for epic feats and I can't imagine a max STR assassin is going to be hurting for Discipline, especially if they have Bullheaded already. Cleave, Great Cleave and Bullheaded, however, are all incredibly useful feats and give a lot to the subrace. To sum up, it seems that out of 8 feats, 3 are useful (Cleave, Great Cleave, Bullheaded), 2 are suboptimal (WP: Exotic, ESF: Parry), and 3 are virtually useless (ESF: Discipline, Armor Proficiency Heavy/Medium). For a build that makes use of WP: Exotic and ESF: Parry, you might have 5 worthwhile feats, but in all likelihood, you're not going to be getting as much use out of them as Banshrae will be getting out of their feats. Speaking of Banshrae, it's important to note that all 5 of their feats are incredibly useful. Sure, it practically requires you to use Assassin Dagger, but if you're going DEX assassin, your damage is already going to be lackluster so you may as well invest as much as possible in what you do best, which WF/GWF: Assassin Dagger helps facilitate. To me, the most obvious suggestion for what to change as far as Half-Dogai feats are concerned is to drop the total number of feats to 5, and give them Cleave, Great Cleave, Bullheaded, Great STR I, and Great STR II.
Obviously, this isn't the most in-depth analysis, but I think it might provide some insights. Provided I've managed to present something akin to a coherent argument, I believe Half-Dogai would be better formulated as follows:
"Half-Dogai" (secret eXtremely Rare)
- FCs: Fighter, Barbarian / BC: Assassin
- STR +8, DEX +2, CON +2, WIS -2, INT +4, Hide +8, Move Silently +8
- Free Feats: Cleave, Great Cleave, Bullheaded, Great Strength I, Great Strength II
- Other: +1 die to Caustic Weapon
This formulation seems, to my mind, to be a lot more straight-forward, lacking the numerous superfluous feats of the original, and geared towards helping STR assassins do what they do best. To my mind, the Great STR feats are necessary addition to the subrace because they free up some much-needed epic feats. You're already going to be swapping your Epic Reflexes feats for Great STRs, devoting all of your Assassin bonus feats to that purpose, since Assassin doesn't offer Epic Prowess, Armor Skin, EWF, OC, or DC as bonus feats. This would allow STR assassins to take for their epic feats something like: Armor Skin, EWF, OC, DC, Epic Prowess, and a couple of Great Strength feats. Furthermore, this current formulation is also a nice upgrade from Half-Molydeus.
How does this look, T Scott? Something along the lines of what you were thinking, or do you think it needs more work?