The mineral website is a hobby that I do when I am not out doing consulting for petroleum geology. In answer to your questions I'll explain how I used the programming tools to create the mineral website. The actual mineral database is in Microsoft Access. This is where I input the basic data. To produce the mineral website, I use MS access queries, forms and/or reports for exporting the mineral data out of access in a text format that is usable by SAS or for importation into a MySQL table. I have different access reports, forms, and queries for most of the tables in the mineral database. The next step is done using the SAS language. For example, I use SAS to create each of the individual mineral pages using their macro language to format each page as an individual file (all 4,442+ of them). The SAS programming takes care of generating the links, creating the load files for MySQL, and creating the other pages. SAS creates the alphabetical pages, Strunz classification pages, Dana classification pages, and the chemistry pages. When all the pages are created, I use FrontPage 2002 to chase down broken links, provide web continuity, and supply the final formatting.
Major new modification to Webmineral. View interactive crystal structure diagrams using jPOWD Applet.
In March, webmineral added crystal structure diagrams using jmol. All attempts to install the structure diagrams on the species pages failed because jmolApplet.jar is large (600 kb) which resulted in unacceptable download times on webmineral's development server.
I looked for alternatives on the web to display crystal structure diagrams. This is the first major website to use the jpowdjpx.jar applet which comes with jPOWD from Materials Data, Inc. The program generates x-ray pattern and structure simulation by reading crystallographic information files (.cif) and generates a simulated x-ray diffraction pattern from the atomic positions. The output of binary crystal files (*.jpx) from the jPOWD program and the jpowdjpx applet allows web display of crystal structures.
| Dr. Steffen Weber has created a new version of JCrystal applet using the SUN-Java Virtual Machine (JVM). The new applet allows rendering transparent faces which is not possible with the MicroSoft JVM - note, the MS-JVM is no longer available from MicroSoft and only resides on older versions of MicroSoft's operating systems (Windows 2000 and earlier). If you have Java installed\enabled and you cannot view the new applet in the window to the left then go to www.java.com and install SUN's JVM (JRE) software on your computer. |
New addition to webmineral.com. See Webmineral News and view items of interest from the mineral world. The topical items may include news of upcoming auctions and mineral specimen sales from webmineral's advertisers, mineral show reports from other sources on the web.
Upgraded phpAdsNew from version 2.0.4 to version 2.0.8. This upgrade should help advertisers view statistics since the phpAdsNew MySQL tables were repaired and optimized.
Major new addition to Webmineral. View interactive crystal structure diagrams using Jmol Applet.
This capability just recently came available with the publication of the open-source program, Jmol, written in java and the companion java applet for web use. This program replaces Chime and does not require that the user install a "helper" plugin. The ability to visualize and manipulate crystal structures completes the capabilities of webmineral as a mineral database. Users can also download and run Jmol as a stand-alone application.
The generation of these mineral structure required the following:
I am nearly complete in the upgrade of the banner ad rotation software on
webmineral.com from the existing perl code to a new php/MySQL system from
phpAdsNew. The planned change over to the
new banner ad system is July 1st. This will provide users with a better
selection of links should the user desire the opportunity to purchase mineral
specimens from the best of the mineral dealers who sell them on the internet.
The new system has a bunch of advantages. These are:
I have got the site mostly restored to the pre June 1st configuration after
they were able to chown the ownership from root to webmineral. I have
re-configured the webmineral's development server to be in line with the
purenetlistings.com site set-up.
I looked into moving webmineral.com to its own server but the costs would just
about equal the monthly income from the banner ads. The problem is bandwidth. As
the bandwidth approaches 100 Gb per month, I am nearing the end of a shared host
solution. This is why I am really cracking down on people who download the site
using web stripping software.
Hosting sites which advertise unlimited bandwidth and 10 Gb disk space for
$10.00 a month are really not a solution because as soon as you start to use
appreciable bandwidth, they cut you off. You get what you pay for in this
business. Sites that advertise $200.00 / month for your own server generally put
you on a cheap pc clone computer with IDE drives and no hot-swap capability.
This is also not a reliable solution.
On June 1st, my provider,
www.vitalstream.com
handed the site over to one of their business associates,
www.purenetlistings.com. All this without prior warning.
When they did that, they used an old backup version of webmineral that is way
out-of-date. To add insult to injury, I cannot update or delete any files using
ftp. I called them about it last night and tech support said that it will be
fixed right away and he would call me back. I guess the technician was too busy
so I sent an e-mail today outlining the problem and I am still waiting for a
response.
Until this problem is fixed, I am unable to restore the site to its original
functioning.
The mineral count in webmineral is now up to 4,442 after updating all minerals to 2004. The crystallography section has been converted to a php/MySQL database for easier updates in the future. All minerals in the database now have a publication date (See IMA Status on species pages). This has allowed an accurate assessment of the number of non-discredited, valid minerals (1,645) present prior to the establishment of the International Mineralogical Association (IMA) and the New Minerals and Mineral Names (CNMMN) in 1959.
Chemical composition pages for each element have been changed to include elements which are present in the empirical formula but are not present in the chemical formula. The changes are illustrated by the following excerpt from the chemical analysis pages (Titanium) accessed by the periodic table.| %Ti | Mineral Name | Chemical Formula | MW |
| 1.90% Ti | Komarovite | (Ca,Mn)2(Nb,Ti)2Si2O7(O,F)2·3.5(H2O) | 502.73 |
| 1.90% Ti | Watatsumiite ! | Na2KMn2LiV2Si8O24 | 905.03 |
Komarovite example shows the presence of titanium in the chemical formula and Ti is highlighted in red.
Watatusmiite example shows the absence of titanium in the chemical formula and the presence of titanium in the empirical formula. The formula is highlighted in gray.
Mindat.org, Webmineral.com and MinMax websites are cross-linked by thousands
of URL references. Although people who use software programs to download whole
websites appear to be a minority, when they do, it affects all
of our users. Especially if they have a broadband connection to the web.
If you really need to have an off-line copy of webmineral, just purchase the
PHOTOGRAPHIC GUIDE TO
MINERAL SPECIES 2nd Edition which features the data files
from webmineral with thousands of mineral images.
The big problem is created whenever one of these people configure programs (eg.
HTTrack 3.0, Teleport Pro, Wget, WebStripper, Webdup, WebReaper, WebZIP, Offline
Explorer, and etc.) in promiscuous mode to "crawl-all-links". This really
hammers the websites and creates access and response problems for their users.
The resulting files created would most probably fill up the diskspace on most
computers with gigabytes of data.
One solution to this problem is to restrict access (see .htaccess documentation
on www.apache.org) using the IP addresses of the worst offenders. I don't like
to do this, but I have restricted access to webmineral from whole subnets when
the offenders move from machine to machine to get around the blocks I put in .htaccess. Anyone caught downloading data from
webmineral will be banned from access to the species pages and the IP address
sent to the other on-line mineral databases for possible inclusion to their ban
list.
If your address is found in the above list and you wish to have the block removed, then send me an e-mail promising not to download the site and I will remove the block.
Added a new class of calculated data for each mineral species, the Fermion Index and complimentary Boson Index. There are some unusual properties which are linked to these values. These properties are not yet well understood. It has been speculated that these properties may link metaphysical (spirit) and physical (real) worlds. This is pure speculation at the moment.
Jeffrey G. Weissman, co-author of the Photographic Guide to Mineral Species is currently updating his Photographic Guide with new and larger images. Jeff has given webmineral permission to use some of his older mineral images from the current version of the CDROM.
These new mineral images are in-addition-to the 1,475 images already in webmineral's image files assembled from individuals over the last three years. This now makes webmineral the most comprehensive (as measured by number of separate species) picture index site on the web.
Please note: The 1,105 new species added from the Photographic Guide represent some of the rarest and hardest-to-find minerals in the world. These images are the "Good, the Bad, and the Ugly" and are an invaluable reference to anyone who needs mineral data.
There is a whole class of information which can be derived from the data contained in webmineral. Calculated values for various different parameters have been added to the species pages in a section titled "Calculated Properties" along with the more familiar "Physical Properties" and "Optical Properties".
The new calculated variables are:
This upgrade of webmineral.com converted all the mineral pictures to a MySQL/PHP database and reporting system. The previous set-up used static pages and server side includes (SSI). This was not the correct way to handle the image data in a large website. The conversion to php to handle image libraries should help site maintenance considerably. The look-and-feel of the old pages has been preserved with the added benefit that many more images can be easily added in the future.
Future plans are to convert as much of webmineral's content to an on-line database as possible. This will help the update cycle and allow more up-to-date content rather than the batch process done on a periodic basis.The mineral count in webmineral is now up to 4,339. This update includes all the error corrections and additional information that have accumulated in the Access database over the year. The chemical analysis have been error checked by converting the chemical compositions to oxides and comparing the sums of oxides to the original analysis.
The most important change with this version of webmineral is the use of cascading style sheets to format certain aspects of mineral data. Please note that the rotational inversion axes of 3 (3BAR), 4 (4BAR), and 6 (6BAR) and overbar hkl values may not be visible to older browsers (Netscape v4 and Internet Explorer v5) which don't use cascading style sheets (.css). This means that 15% of webmineral's users cannot view these changes.
|
# of Species |
Remarks |
| 179 | Not Approved by the IMA. |
| 16 | Formerly valid species Discredited by the IMA. |
| 118 | Proposed new minerals awaiting publication. |
| 12 | Duplicate minerals with valid Dana or Strunz Classification Numbers. |
| 4,014 | Valid Mineral Species Approved by the IMA or considered Valid prior to 1959 (Grandfathered species). |
| 4,339 | Grand Total |
| 6,300 | Number of synonyms of mineral names (Grandtotal=10,639) |
Due to technical problems with webmineral's provider, vitalstream.com, the site was unavailable from Saturday, August 9, till Monday morning, August 11. We are examining the problem to prevent a reoccurrence. Thank you for your patience.
New Feature. The index page has been modified to show random User Tips and Factoids. This list is summarized in this link. If you want to add to this list, please e-mail suggestions to the webmaster.
Preparations for a major site update with new and revised data is underway. As part of the update, mineral reflectivity data for hundreds of opaque species were added. Webmineral now has the largest collection of spectral opaque mineral reflectivity data on the web. This reflectivity data can be accessed along with the other data types on the mineral species pages (See Reflectivity). This data is also summarized in the determinative mineralogy tables for opaque minerals.
New Feature. Need a fast and easy mineral label for your specimens? Just visit the species page for the particular mineral you need a label for and print a handy copy to place under your specimen. The label (already filled in with necessary data) is included in the "See Also" section of the species page.
For more options in creating mineral specimen labels, you can take the time and download Ososoft's Mineral Label 5.0 for free. Bob's Rockshop is a mirror site so you can also download it there.
The mineral count in webmineral is now up to 4,281 after adding 61 new minerals and deleting 35 obsolete entries. This update includes all the error corrections and additional information that have accumulated in the Access database over the year.
The big news is webmineral now has a working version of Gladstone-Dale Compatibility Index programmed in SAS and the data is now available on the species pages. Many thanks to Dr. Anthony Kampf of the Los Angeles Museum of Natural History who provided the background information and a copy of his ChemComp program which does the calculation. The reference for this technique is Mandarino, J. (1981) paper in Can, Min, (1981), Vol 19, pp 441-450).
Basically, the compatibility index (CI) compares the Gladstone-Dale constants (KC) for the chemical analysis using the oxide values for each mineral and (KP) derived from density and index of refraction values. The CI is a measure of the "goodness" of the data based on physical and chemical properties for each mineral. An additional benefit is that the index of refraction can be calculated (from KP) if the optical data are not present.
The application of this technique pointed out deficiencies in the mineral data. The results are as follows:
These have been corrected and work is underway to correct some of the deficiencies in the data when the oxide percentages are derived from the empirical formulas (∑ ± 100% ). The tolerance is 95% to 105% of the sum of oxides.
Webmineral has obtained a copy of Dana's 7th edition. All those wonderful crystal drawings missing in modern mineralogy references are now in the process of being added to webmineral using Jcrystal. It is a slow process so I can only do a few at a time.
In addition, I have also purchased a copy of "Determinative Mineralogy and Blowpipe Analysis" by Brush & Penfield, 1906. This book, first published in 1874, is the state-of-the-art manual for qualitative blowpipe analysis of chemical elements in minerals. I am adapting some of the element tables (see Borax and Phosphate) and the analytical techniques and have been re-doing portions of the chemical element pages.
This update now has the chemical composition of all minerals in place. The elements are calculated as % and as oxide % where applicable. This update also includes the new minerals for 2000 (4,255 total). The main changes are as follows:
I am trying out a new machine translation service from InterTran to replace the Babelfish service that I was using. It provides a larger selection of languages and the Java applets work after the translation. If there is enough of a demand for this service, I could upgrade the service from a free translation to a paid-basis translation. The paid-basis translation would allow the selection of more appropriate grammer for the Mineralogy Database and would run on a dedicated server for faster translations.
I am moving the Mineralogy Database over to a new server. My old ISP wanted to charge me an arm and a leg to host the mineralogy database site because of its size.
The temporary URL is http://webmineral-com.temp-url.com/default.shtml until the domain name is transferred. The main change is from a NT server to a UNIX server. The old server was just too overloaded with site bandwidth running at 22 Gb/month and the cgi scripts were corrupting the counter files on a regular basis. Plus, microsoft does not recommend running cgi scripts on a NT domain server.
I intalled a new search engine on the webmineral.com site. For a preview, see ../search.shtml. It is a shareware cgi perl script system from fluid dynamics that allows administration from web browser pages and it seems to work very very good. It is much better than Microsoft's search engine supplied with FrontPage.
I have submitted the domain change to registrar.com. When the changes propagate though the DNS system, webmineral.com should be active on the new servers at http://www.publichost.com. Hopefully, the changes should be transparent to the users of the Mineralogy Database. If there are any problems, please notify dbarthelmy@webmineral.com
There are file corruption problems with the cgi programs on the Mineralogy Database's server. Please bear with me until I change the site over to a server that can handle the load.
Changed the "See Also:" section on the mineral pages. Marc Farve of Alkali-Nuts and Herb Yeates of Franklin Minerals have changed their access to mineral data and the reference URL's have changed. The "see also" section of the mineralogy database is created from a listing and I can change it at any time easily because I use SSI includes for the data.
I have also expanded the search engine links in the "see also" section to include most of the currently available search engines on the web. This is really handy because it allows specific data to be called up by just clicking on the link.
Added a redesigned advertisers page to allow clients of the mineralogy database to log-on and review their usage statistics in real time.
There are more images being added to the site with photos generated by the site's sponsors. These photos are exceptional in quallity and may be viewed on the specimen page in the database. The images are heavily cross-link to the data pages and vice versa. I am adding more images on a regular basis since the advertising revenue allows for the increase in the size of the site.
The site has gone through a major update. One of the biggest changes is the inclusion of .shtml pages for the individual mineral pages. Unfortunately, most sites that link to the individual mineral species page are now out-of-date. This means those sites who have linked http://web.wt.net/~daba/data/Quartz.html or ../data/Quartz.html will now need to link to ../data/Quartz.shtml.
Lets go on with the additions:
The data pages on the old site (http://web.wt.net/~daba/Mineral) have been deleted. Please use ../ as the URL for the Mineralogy Database. I have also added advertising banners to help support the increased costs associated with this move. So if you like this site, please visit the sponsors who are kind enough to place their ads in the Mineralogy Database.
I hope I have left enough redirection pages and .htaccess files on the old site so all those people who link to the old site can find the new site. If you have any questions, complaint, gripes, and etc. about this move please let me know (dbarthelmy@webmineral.com).
I am over my quota of disk space on the wt.net personal account at http://web.wt.net~/daba/Mineral. I guess this means I will have to go commercial (http://www.webmineral.com) with the database. Please bear with me until I can transfer to the new domain.
The internal layout of the database will remain the same, only the domain name will change
I am going to keep all the main pages on the old site and link to the new site when I make these changes. The pages I will keep in the old site are
http://web.wt.net/~daba/Mineral/index.htm http://web.wt.net/~daba/Mineral/chemical.html http://web.wt.net/~daba/Mineral/X-Ray.html http://web.wt.net/~daba/Mineral/crystall.html http://web.wt.net/~daba/Mineral/danaclass.html http://web.wt.net/~daba/Mineral/strunz.html http://web.wt.net/~daba/Mineral/determin.html http://web.wt.net/~daba/Mineral/specimens.html http://web.wt.net/~daba/Mineral/Alphabetical_Listing.html http://web.wt.net/~daba/Mineral/other.html http://web.wt.net/~daba/Mineral/dana/newdanasort.html
This will avoid having the site go dead for people who just link those pages. If you have any questions or concerns, please feel free to ask.
Added a search engine feature from FreeFind. I have not tried it out extensively, so have at it and report back your impressions. The mineralogy database site was too large for their spider to index fully so there may be some holes in this feature. It will not do element searches for "Ca Fe SO4" but it will find minerals where the chemical analysis are present (not present on all minerals in the database). For example, try a search on "sodium iron monoclinic". It does a good job of finding all the minerals with "Canada" in the locality section. Try "JCrystal" and you will get all the pages with Steffen Weber's JCrystalApplet on them.
Steffen Weber offered the use of his JAVA based, JCrystalApplet for the webdisplay of crystal shapes (see the help file for a description). This program allows a user to manipulate a crystal shape with mouse and keyboard control with a realism almost equal to actually holding a real crystal model. It is such a nice program that I added the JCrystal version to the crystallography table for interactive examples of crystal morphology for the different crystal systems. I left the original static images in place so that older browsers without Java support could view the content.
Steffen also offered me the use of a beta copy of the actual stand-alone software, JCrystal, which creates the html files that JCrystalApplet uses. Using this program, I have started adding crystal morphology images to some of the pages (e.g., quartz, benitoite) for the more common minerals as an added feature to the mineral database. If there are any people out there who use this program to create crystal shapes for the different minerals, I would appreciate a copy of the hkl files so I could add that mineral to the ones already in the database.
This is a major update involving every page of the Mineral Database. Because of the many items that were changed, I decided to list those in a table.
Revised the crystallography table to include examples of crystal morphology for the different crystal systems. This fixes one of the weakest links (content wise) in the mineralogy database. I have also placed a prototype of a new layout of the mineral pages (see example) for testing on different browsers. When it is field tested, then I'll update the whole site with new entries in the database.
Added a statistical web tracker to monitor the usage on the mineralogy database. This tracker does not keep individual statistics, just general information. I was kind of interested in knowing who was using the database, who is linking to it, and how many new users were accessing the information. It is open to public access by clicking on the ExtEME tracking logo on the bottom of the index page.
Revised the links page to include more sources of mineral data on the web or from commercial sources.
Added Dana's New classification. This arrangement of mineral species is based on Dana's New Mineralogy by Richard V. Gaines, H. Catherine Skinner, Eugene E. Foord, Brian Mason, and Abraham Rosenzweig, (ISBN: 047119310-0) Copyright © 1997, John Wiley & Sons, Inc. This material is used by permission of John Wiley & Sons, Inc. Thanks to Wiley's for granting permission.
Abraham Rosenzweig graciously sent me some unpublished material dealing with the silicate classification. He writes, "A detailed explanation of the logic behind the development of the class numbers in this group was to have been part of the new book, and manuscript for this was furnished to the publisher. However, the publisher chose to eliminate this section, along with some other matters, in order to save pages, as the book was already large." I have incorporated this data in the Dana classification pages.
Took pictures of my personal mineral specimens and added a page of images. I used a 200mm telephoto lens with macro focusing to take the pictures and scanned them after the photo lab was finished processing them. The images were enhanced using Adobe PhotoShop 4.0 and resized to 640 x 480 resolution.
Created the precursor program for the Mineralogy Database using the data files from Microsoft Access to create a Win95 Hypertext Help file using RoboHelp95.
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
