Monday, June 4, 2012

Why David Peters' analysis sucks

For a while now, Peters has played the part of the outcast whose new ideas have been shunned by the outdated but stubborn establishment, locked in its consensus purely due to tradition and an unwillingness to include numerous genus-level OTUs in an analysis.  And he actually has a point.  Published large scale reptile analyses DO suffer from using suprageneric OTUS, are heavily miscoded, and include way too few taxa to be useful.  However, Peters' analyses are even worse.  One huge reason is that they include what are certainly wrong codings caused by Peters' method of running photos through his computer to "find" things people looking at the specimen missed, like the entire posterior skeleton of Longisquama, babies and crazy frills on it and various pterosaurs, teeth in toothless taxa, and anything else you can imagine.  But even ignoring that, there's another level at which Peters' analysis fails, and that's when we look at the characters themselves.  I asked Peters for his matrix today (and I will credit him for being so transparent as to send it to anyone who asks, despite his study not being published in peer reviewed literature), and here's what I found...

Let's just look at dental characters.  Character 107 is palatal teeth present vs. absent, but character 96 covers vomer teeth being absent and character 101 covers pterygoid teeth being absent, so this is correlated right off the bat.

Character 108 is number of premaxillary teeth with the states four, more than four, one two or three, none, teeth fused and three or four plus medial.  Ack, where to begin?  First, no character is ordered, so while "four" should be between "1-3" and ">4", it's not.  Second, multiple variables are involved here.  Each character should only involve one variable.  Having medial premaxillary teeth should be its own character.  Having teeth fused should also be its own character.  And since we don't know toothless premaxillae evolved via decreasing number of teeth, a toothless premaxilla should also be its own character.  Any coding not falling within the states given should be inapplicable (e.g. Citipati for number of premaxillary teeth 1 vs. 2 vs. 3 vs. 4. vs. 5 vs. 6 vs. 7 or more).  A good way to test for how well character states are defined is to see if any possible morphologies couldn't be coded.  In this case, what if a taxon had six premaxillary teeth with medial teeth too?  Or what if it had less than four fused teeth?

Character 109 is medial premaxillary teeth larger vs. smaller than lateral teeth.  And yet all dinosaurs are coded 0, when none have medial premaxillary teeth.  They should be coded inapplicable.  This is the same for many derived reptiles, no doubt.  I notice Thecodontosaurus is coded as derived though, when no premaxilla is known, and even Pantydraco only has lateral teeth.

Character 110 is premaxillary teeth robust vs. tiny to absent.  Yet absent was already covered by character 108, and these states are never defined so that I could always know when a specimen has state 0 or 1.

Character 111 is procumbant premaxillary teeth, which is fine except needing an angle at which teeth are defined as procumbant.

No complaints about character 112- maxillary tooth length less than twice root depth (presubamly FABL).

Character 113 is canine maxillary teeth, but Peters codes taxa with toothless maxillae such as Hupehsuchus and Endennasaurus as lacking canine maxillary teeth when they should be inapplicable. 

Character 114 is the position of the last maxillary tooth compared to the orbit, but is not ordered, so "mid orbit" is not between "anterior to orbit" and posterior to orbit".  Furthermore, it includes a state for "toothless", which makes it correlated with the next character...

Character 115 is maxillary teeth sharp vs. blunt vs. multicusped vs. multicusped with constricted bases vs. absent vs. constricted bases.  This makes 108 look formatted well.  SO MANY variables are combined!  Sharpness, number of cusps, constriction of base, presence.  This needs to be four characters.

So yeah, that's just a sample.  Can you all see why no one trusts Peters' analysis, even ignoring his coding accuracy in the first place?  It's true some other analyses have some of the same issues, like Rieppel's (http://theropoddatabase.blogspot.com/2011/06/rieppels-reptile-matrix-and-turtle.html), but this doesn't make Peters' more likely to be correct.  Honestly, Rieppel's, Mueller's and Peters' are all flawed.  I don't trust the results of any of them.  If someone wants to be the true shining light on the subpar tradition of reptile analyses, they need to understand how to construct characters well, and not put their faith in graphic artifacts created by Photoshop.  Peters can accomplish the first step by posting this on his blog and adusting the characters as per my critique.


3 comments:

  1. I recently wrote Mickey, hoping for a reply after the request for the characters. Mickey pointed me toward this blog as an answer. Here is my reply copied from my email.

    Re: Any thoughts yet on the tree?

    I understand.
    But it works. All sister taxa look alike and one can trace the gradual development or loss of traits in a series of ancestral taxa in every case.
    Thanks for your thoughts, Mickey.

    Your choice of words could be improved. "Sucks" comes off the street, which is inappropriate for scientific discourse. Blogging first without discussion is also bad practice in this case. It shows you can't be trusted. I'm sorry you chose to go this way.

    BTW the medial teeth in character no. 109 are the ones closest to the midline suture, the ones most medial. In character no. 108 there are indeed taxa with a medial tooth, so I can see your confusion here.

    With regard to correlated characters, it's quite hard not to correlate most characters. We already know, for instance, that large numbers in the vertebral count are correlated with small to absent limbs. Just a for instance.

    How do you "trust" an analysis? Don't trust it. Run a competing one. If an amateur like me can do it, surely any professional can. And of course, as above, if one can trace a gradual accumulation of traits without too many aberrant autapomorphies, that's another good sign.

    I also challenged you to list any genus that was mis-nested and find a better nesting closer to another genus as the beginning of a discussion. Suprageneric taxa, of course, cannot be used They're too nebulous.

    Finally, because Science is the search for the truth, very rarely does one "find" the truth. We only hope to get closer with each test and its improvements. The large reptile tree takes us that one step closer. I'm finding errors and making improvements weekly, especially when new taxa are introduced. And I'm looking forward to someone, such as yourself, taking it to the next level.

    Presently with 223 characters and 305 taxa (not counting the pterosaur tree and the therapsid tree, that comes to 68,000 matrix scores, less a certain number for headless and body-less taxa. Each score affects every other one. With such a number recovering a single tree, it's going to be hard to change the topology on a large scale.

    I'm not asking you to trust my tree. Test it and test the results. For many, trust issues go back to tradition (which are by definition trusted), affinity (professors have more of this than amateurs) and novelty (which is rarely trusted...at first.)

    Dave

    ReplyDelete
  2. Fair enough having the email conversation continued here. For those following, here's what I wrote Peters that he's replying to here- "The characters themselves are too flawed to make adding or deleting taxa, recoding the matrix or constraining topologies useful. As I note in that post though, Rieppel's traditional reptile matrix suffers from similar issues, though to a lesser degree. If you have any questions on how to design character states properly, feel free to ask. Basically, every character should only score for one variable. If you have multiple states, and some are intermediate (in size, number, angle, etc.), make the character ordered. Fix it up, then we'll see what happens."

    You say that despite these problems, your analysis works. But how do you know? For fossil taxa, we have no independent method of comparison, and your reptile tree is different from both nuclear and mitochondrial results for living squamates. Your mammal results are also different from genetic analyses if I recall. You say every case has a gradual development or loss of traits, but what about... oh... hyposphene-hypantrum articulations in the dorsals? Your tree doesn't support Saurischia, so it seems like something that might perform worse in it. Traditionally, besides various crurotarsans, it evolves in Saurischia and is lost much higher up in a few clades (birds, some titanosaurs, parvicursorines). In your tree, it has to be lost in Marasuchus, silesaurids and Ornithischia, in addition to those derived clades. I went to check your tree to see how you coded it, but it's not even included. More on this later.

    Skipping over your tone argument... you said on your blog (http://pterosaurheresies.wordpress.com/2012/06/02/reptile-tree-experiments/#comment-1153) "Where’s your tree? Where’s my flawed data? So far you’re just repeating untested tradition and sniping from the sidelines. Please be specific and scientific." So you can't very well now criticize me for posting on your flawed data without discussing it with you first.

    As far as medial teeth go, I retract that criticism. My new criticism is to call these mesial or even anterior teeth to avoid confusion.

    I agree that many characters end up co-occuring often in taxa. What we have a duty to do as phylogeneticists is to eliminate characters that logically entail other included characters. Large numbers of vertebrae CAN occur without limb reduction, such as in elasmosaurs. Thus the characters are not correlated in the sense we care about. But you can't lack vomer teeth without also lacking palatal teeth, even hypothetically. So you can't have these as separate characters without coding some taxa inapplicable.

    ReplyDelete
  3. As for how to trust an analysis, you look for character and taxon number for a start. Yours is great for the latter but so-so for the former (only 99 postcranial characters across all Amniota?!). Then you see if the characters are formed well, which as I've shown, they are not. Then you see if taxa are accurately and completely coded. For the former I'm doubtful of yours due to your Photoshop method, but a glance at the matrix seems to show you were good about coding everything you thought was possible. You also check to see if characters supporting alternative topologies were included, and here I'm going to provisionally say your matrix fails. We'll see in my next post when I look at what Saurischia synapomorphies you've included. A final method is to check the CI, and I gotta say at least you haven't planned your matrix to only include characters that support your topology. Your CI of .1040 is impressively low.

    I already told you taxa that were misnested, limiting myself to dinosaurs, since I'm most familiar with that clade- http://pterosaurheresies.wordpress.com/2012/05/31/celebrating-300-taxa-in-the-large-reptile-tree/#comment-1152 .

    Look David, I don't think you're a bad person. You have noble goals, and in some respects (like including numerous genus-level OTUs) you're ahead of the pros and absolutely right to criticize them. But there's a reason the pros haven't done what you and I would like them to- it's a HUGE project. 223 characters is nothing compared to what you'd need for this analysis. At least a thousand, and we'll see why in my next post. But more than that, you need understanding. I just specialize in theropods, and I'm still finding out what certain characters mean and when the traditional knowledge about a coding is wrong. Turner in his 2008 thesis thought the obturator tuber of dromaeosaurids and birds was the same as the obturator process of avetheropods. Senter et al. in the Yurgovuchia paper thought the flat internarial bar of ornithomimosaurs, alvarezsaurs and troodontids was the dorsal curvature in lateral view as opposed to the cross section. These are professionals. To learn the details of all amniotes well enough to create and code them for an analysis would take decades. You or I could improve on Rieppel's or Mueller's just because they are so far from ideal, but neither of us could do the job justice.

    ReplyDelete