Besides agreeing with the other answers, asbestos can be found as inclusions in other minerals such as quartz.

Asbestos is created from fibeous minerals of varying sorts found around the world. There is far more info in the linked wikipedia article below, which seems very similar to the previous answer…

Asbestos (a misapplication of Latin: asbestos “quicklime” from Greek ἄσβεστος: a, “not” and sbestos, “extinguishable”) describes any of a group of minerals that can be fibrous, many of which are metamorphic and are hydrous magnesium silicates. These minerals, together with their occurrences, uses, and associated hazards, have been discussed in detail by Guthrie and Mossman (1993).

The name is derived for its historical use in lamp wicks; the resistance of asbestos to fire has long been exploited for a variety of purposes. Asbestos was used in fabrics such as Egyptian burial cloths and Charlemagne’s tablecloth (which according to legend, he threw in a fire to clean). Asbestos occurs naturally in many forms (see below); it is mined from metamorphic rocks.

When asbestos is used for its resistance to fire or heat, the fibers are often mixed with cement or woven into fabric or mats. Asbestos is used in brake shoes and gaskets for its heat resistance, and in the past was used on electric oven and hotplate wiring for its electrical insulation at elevated temperature, and in buildings for its flame-retardant and insulating properties, tensile strength, flexibility, and resistance to chemicals. The inhalation of amphibole asbestos fibers, however, can cause a number of serious illnesses, including cancer. Many uses of asbestos are banned in multiple countries.

Pretty much from rocks