Even though I'm not super religious I find some sections of the Bible fascinating.  Is there much verifiable historical records that follow the bible?
What is regarded as canonical text differs depending on traditions and groups; a number of Bible canons have evolved it is a very intetresting thing to read 
Oh I've heard that some books the Catholic church decided not to include in the Bible.  Do you know which books were left out or removed at some point in the canon?
im not really sure but i know that , The Christian Old Testament overlaps with the Hebrew Bible and the Greek Septuagint
The Septuagint is new to me.  Does the bible have any other particular unusual history associated to it? 
yeah The New Testament is a collection of writings by early Christians, believed to be mostly Jewish disciples of Christ
As a relevant segue, Mormonism is based of the bible, but it's notably different from most established forms of christianity.  Does the Mormon bible have any differences, like is there a third testament?
i am not really sure, although i have read the old testament couple of times