  
| Author: | Locutis  | | Posted: | Dec 11, 2010 11:38 | | Subject: | Re: Stop purging data | | Viewed: | 65 times | | Topic: | Suggestions | |
|
| Our internal work server (used for invoicing, accounting, etc.) has only 5 users.
However, our external webserver (online store, auction site, gold/silver bullion
quotes, etc.) receives many millions of hits per month. We send over 60Gb of
html data over our fibre internet connection every month.
Our customer database consists of 80,000 + active customers.
On our external server (same hardware as the internal, 2.8 GHz processor, 2x500
GB RAID 1 hard drives) we operate a standard webserver, an auction website (with
a current catalog of 4,000 active items, and 40,000 archived items) which accepts
and processes bids in realtime, an online store with over 2500 active items,
and a gold and silver bullion website which provides realtime quotes to 10's
of thousands of customers every minute.
Our website is high profile as well. I am one person programming in "spare time",
and have made all of this work, and work efficiently.
Cameron
In Suggestions, AggieSava writes:
| | In Suggestions, locutis writes:
| | In Suggestions, B0RIS writes:
| | In Suggestions, Timothy_Smith writes:
| | My suggestion: stop purging data.
It's the 21st century, mass storage is cheap.
There's no at all reason to purge data ever.
|
Data purging is not done to save storage cost. It is done to cut data access
time. My guess is all bricklink visitors are served by only 1 processor.
Boris.
|
I don't know how Bricklink works, but I know on my work server, I can search
through AND display over 20,000 records in less than 1 second. We have one single
server, with 2x 500GB hard drives on RAID 1 (which means they duplicate the data
over 2 drives), running Apache2, with a 2.8 GHz Pentium processor.
If I want to search the "archived data" which takes 10,000 records and magically
combines it into 1 for speedier access (I programmed it to archive records over
2 years old 10,000 at a time into one database entry), there's 200,000 records
to search, it takes only several seconds to search, compile, and display the
information.
Again, I don't know how Bricklink works, but I'm far from a professional programmer,
and I made the site at work operate efficiently this way. We never purge ANY
data, and have access to all data going back to 1997 when we started computerizing.
I'm one single computer person at work, and computer work isn't even my job,
I do it in my "spare time" while trying to manage the company.
Cameron
|
But how many people are using your work server? There are 152,351 registered
members of Bricklink as I'm writing this, and Bricklink already has an extensive
database to go through as it is for every single person. I can imagine how if,
as suggested, BL has only one processor, a database containing all the orders
and all the forum posts since the beginning would slow considerably for all these
users.
But as also already suggested, hardware is getting cheaper and cheaper.
--Tony
|
|
|
Message is in Reply To:  Re: Stop purging data - AggieSava (992)  | [...] But how many people are using your work server? There are 152,351 registered members of Bricklink as I'm writing this, and Bricklink already has an extensive database [...]
(164 months ago, Dec 11, 2010, to Suggestions) |
Message Has 1 Reply: Re: Stop purging data - Locutis (69) | I forgot to mention something. Last year I looked forward towards the future, and investigated High Availability Linux for our servers. It is actually quite easy to setup and [...]
(164 months ago, Dec 11, 2010, to Suggestions) |
19 Messages in this Thread: Msg 1 - Timothy_Smith (1537) 164 months ago Dec 11, 2010 to Suggestions
 Msg 2 - legomadsteve (72) 164 months ago Dec 11, 2010 to Suggestions
 Msg 3 - redbeardlegoman (83) 164 months ago Dec 11, 2010 to Suggestions
 Msg 4 - JoeMomma (1214) 164 months ago Dec 11, 2010 to Suggestions
Msg 5 - bb166186 (89) 164 months ago Dec 11, 2010 to Suggestions
Msg 6 - Timothy_Smith (1537) 164 months ago Dec 11, 2010 to Suggestions
Msg 7 - Locutis (69) 164 months ago Dec 11, 2010 to Suggestions
Msg 8 - AggieSava (992) 164 months ago Dec 11, 2010 to Suggestions
Msg 9 « - Locutis (69) 164 months ago Dec 11, 2010 to Suggestions
Msg 10 - Locutis (69) 164 months ago Dec 11, 2010 to Suggestions
Msg 11 - Reki_Lobsheek (2465) 164 months ago Dec 11, 2010 to Suggestions
Msg 12 - Locutis (69) 164 months ago Dec 11, 2010 to Suggestions
Msg 13 - matthewcrandall (83) 164 months ago Dec 11, 2010 to Suggestions
Msg 14 - Brickwilbo (1534) 164 months ago Dec 11, 2010 to Suggestions
Msg 15 - Rbobo (3014) 164 months ago Dec 11, 2010 to Suggestions
Msg 16 - wahiggin (2868) 164 months ago Dec 11, 2010 to Suggestions
Msg 17 - eileenkeeney (1610) 164 months ago Dec 12, 2010 to Suggestions
Msg 18 - tomte (82216) 163 months ago Jan 30, 2011 to Suggestions
Msg 19 - BLUSER_228233 (114) 163 months ago Jan 30, 2011 to Suggestions
Entire thread on one page
This message and all its replies on one page
|
Canada, Manitoba
