We each enjoy what we each enjoy.
For me I like the tradition of Division winners and would sooner throw out the entire wild card system...
But if Record was to supersede Division winners I am fine with that if we go all-in on the concept...I personally see Conference as being the same historical notion as Divisions and I see no reason to not just take X teams and line them up in the payoffs by record if the goal is to get the teams with the best record.
If winning a Division isnt the most significant thing in making the playoffs I really dont see why it should be significant at all. If a team is getting an unfair shake because it has a better record than a team that doesnt get a home playoff game than it should also follow that a team getting left out of the playoffs in favor of a division winner (even in another conference) is getting worse of an unfair shake.
It seems to me you are only taking the premise of "the best" teams half way and yielding to some level of tradition. Which to me personally doent make much sense.
Some historical notes
Originally there were no playoffs only record (win% particularly) mattered..with some uneven scheduling it lead to thinks like the Pottsville Maroons getting jobbed out of a championship.
In 1933 they created 2 Divisions and those division champions played in the championship game..originally 5 teams per division. Worth noting in 1934 the very next season the Giants played in the Championship game and the Lions with a better record did not because Division standing was the key..The Giants won the Championship.
In 1950 the AAFC folded and 3 teams were added to the NFL...the 2 Divisions went to 6 teams each and resulted in tied records in both divisions which lead to the first set of playoff games. A similar scenario occurred in 52 and again in 57 and 58. In 1957 the Lions were the beneficiary and ended up winning the NFL Championship after winning their playoff game.
The AFL started in 1960 and by 1965 there were 7 teams in each NFL Division and they once again had a playoff game for the Western Division Colts vs Packers
The 1st Superbowl was in 1966 and even then there was only a single championship game in the "NFL" played only by the division winners. (followed by the superbowl)
Note in 1967 which the NFL went to 4 Divisions of 4 teams each it created the aforementioned odd man out scenario. EVen though the Colts got jobbed they stayed with the 4 Division winners until the AFL merged in 1970.
Worth noting is that Homefield advantage at the time was not merited by record but was purely rotational. This eventually lead to the famous Ice Bowl, partially because before it occurred Green Bay with an inferior record hosted a home game against the NFL regular season team with the best record the Rams.
In 1970 they only had 3 Divisions in each conference and thus created the Wild Card team. The divisions were still small with 4 or 5 teams each.
An extra wild card team was added in 1979.
In 1982 they had a strike shortened season so that lead to some anomalies in the post season.
In 2002 because of expansion teams being added and because they specifically wanted smaller Division they realigned the NFL back to 4 Divisions per conference.
Since the playoffs were created in the NFL Divisions and Conference have always been the Priority. If we are going to scrub it because now Record is more important... I can understand that...but then I say scrub the Conferences as well because they are simply there for the same traditions as the Divisions are there for. Conferences should not be an exemption if Records is truly the indicator of best teams.