Irregular military
Although they are part of a regular army, United States Special Forces are trained in missions such as implementing irregular military tactics. However, outside the United States, the term special forces does not generally imply a force that is trained to fight as guerillas and insurgents. Originally, the United States Special Forces were created to serve as a cadre around which stay-behind resistance forces could be built in the event of a communist victory in Europe or elsewhere. The United States Special Forces and the CIA's Special Activities Center can trace their lineage to the OSS operators of World War II, which were tasked with inspiring, training, arming and leading resistance movements in German-occupied Europe and Japanese occupied Asia.
In Finland, well-trained light infantry Sissi troops use irregular tactics such as reconnaissance, sabotage and guerrilla warfare behind enemy lines.
The founder of the People's Republic of China, Mao Zedong actively advocated for the use of irregular military tactics by regular military units. In his book On Guerrilla Warfare, Mao described seven types of Guerilla units, and argues that "regular army units temporarily detailed for the purpose (of guerilla warfare)," "regular army units permanently detailed (for the purpose of guerilla warfare)," and bands of guerillas created "through a combination of a regular army unit and a unit recruited from the people" were all examples of ways in which regular military units could be involved in irregular warfare. Mao argues that regular army units temporarily detailed for irregular warfare are essential because "First, in mobile-warfare situations, the coordination of guerilla activities with regular operations is necessary. Second, until guerilla hostilities can be developed on a grand scale, there is no one to carry out guerilla missions but regulars." He also emphasizes the importance for the use of regular units permanently attached to guerilla warfare activities, stating that they can play key roles in severing enemy supply routes.

Effectiveness: While the morale, training and equipment of the individual irregular soldier can vary from very poor to excellent, irregulars are usually lacking the higher-level organizational training and equipment that is part of regular army.  This usually makes irregulars ineffective in direct, main-line combat, the typical focus of more standard armed forces.  Other things being equal, major battles between regulars and irregulars heavily favor the regulars.
However, irregulars can excel at many other combat duties besides main-line combat, such as scouting, skirmishing, harassing, pursuing, rear-guard actions, cutting supply, sabotage, raids, ambushes and underground resistance.  Experienced irregulars often surpass the regular army in these functions.  By avoiding formal battles, irregulars have sometimes harassed high quality armies to destruction. 
The total effect of irregulars is often underestimated.  Since the military actions of irregulars are often small and unofficial, they are underreported or even overlooked.  Even when engaged by regular armies, some military histories exclude all irregulars when counting friendly troops, but include irregulars in the count of enemy troops, making the odds seem much worse than they were.  This may be accidental; counts of friendly troops often came from official regular army rolls that exclude unofficial forces, while enemy strength often came from visual estimates, where the distinction between regular and irregular were lost.  If irregular forces overwhelm regulars, records of the defeat are often lost in the resulting chaos.

History: By definition, "irregular" is understood in contrast to "regular armies", which grew slowly from personal bodyguards or elite militia. In Ancient warfare, most civilized nations relied heavily on irregulars to augment their small regular army. Even in advanced civilizations, the irregulars commonly outnumbered the regular army.
Sometimes entire tribal armies of irregulars were brought in from internal native or neighboring cultures, especially ones that still had an active hunting tradition to provide the basic training of irregulars.  The regulars would only provide the core military in the major battles; irregulars would provide all other combat duties.
Notable examples of regulars relying on irregulars include Bashi-bazouk units in the Ottoman Empire, auxiliary cohorts of Germanic peoples in the Roman Empire, Cossacks in the Russian Empire, and Native American forces in the American frontier of the Confederate States of America.
One could attribute the disastrous defeat of the Romans at the Battle of the Teutoburg Forest to the lack of supporting irregular forces; only a few squadrons of irregular light cavalry accompanied the invasion of Germany when normally the number of foederati and auxiliaries would equal the regular legions. During this campaign the majority of locally recruited irregulars defected to the Germanic tribesmen led by the former auxiliary officer Arminius.
During the decline of the Roman Empire, irregulars made up an ever-increasing proportion of the Roman military. At the end of the Western Empire, there was little difference between the Roman military and the barbarians across the borders.
Following Napoleon's modernisation of warfare with the invention of conscription, the Peninsular War led by Spaniards against the French invaders in 1808 provided the first modern example of guerrilla warfare. Indeed, the term of guerrilla itself was coined during this time.
As the Industrial Revolution dried up the traditional source of irregulars, nations were forced take over the duties of the irregulars using specially trained regular army units. Examples are the light infantry in the British Army.

Irregular regiments in British India: Prior to 1857 Britain's East India Company maintained large numbers of cavalry and infantry regiments officially designated as "irregulars", although they were permanently established units. The end of Muslim rule saw a large number of unemployed Indian Muslim horsemen, who were employed in the army of the EIC. British officers such as Skinner, Gardner and Hearsay had become leaders of irregular cavalry that preserved the traditions of Mughal cavalry, which had a political purpose because it absorbed pockets of cavalrymen who might otherwise become disaffected plunderers. These were less formally drilled and had fewer British officers (sometimes only three or four per regiment) than the "regular" sepoys in British service. This system enabled the Indian officers to achieve greater responsibility than their counterparts in regular regiments. Promotion for both Indian and British officers was for efficiency and energy, rather than by seniority as elsewhere in the EIC's armies. In irregular cavalry the Indian troopers provided their horses under the silladar system. The result was a loose collection of regiments which in general were more effective in the field than their regular counterparts. These irregular units were also cheaper to raise and maintain and as a result many survived into the new Indian Army that was organized following the great Indian Rebellion of 1857.

Irregular military in Canada before 1867: Before 1867, military units in Canada consisted of British units of volunteers.
During French rule, small local volunteer militia units or colonial militias were used to provide defence needs. During British control of various local militias, the Provincial Marine were used to support British regular forces in Canada.

Other instances of irregulars: Use of large irregular forces featured heavily in wars such as the Three Kingdoms period, the American Revolution, the Irish War of Independence and Irish Civil War, the Franco-Prussian War, the Russian Civil War, the Second Boer War, Liberation war of Bangladesh, Vietnam War, the Syrian Civil War and especially the Eastern Front of World War II where hundreds of thousands of partisans fought on both sides.
The Chinese People's Liberation Army began as a peasant guerilla force which in time transformed itself into a large regular force. This transformation was foreseen in the doctrine of "people's war", in which irregular forces were seen as being able to engage the enemy and to win the support of the populace but as being incapable of taking and holding ground against regular military forces.

Examples: Arbegnoch -  Guerrilla force in occupied Ethiopia 1936-44.
Armatoloi - Ottoman Greek irregulars
Armenian fedayi – Armenian irregular units of the 1880s–1920s
Atholl Highlanders – The only legal and still existing private army in Europe under the command of the Duke of Atholl in Scotland, United Kingdom, (1777–1783 and since 1839)
Bands - (Italian Army colonial and foreign irregulars)
Bargi - Maratha horsemen 1741-51.
Bashi-bazouk – Irregular mounted mercenary in the Ottoman Empire
Border ruffian / Jayhawker
Bushwhackers – Irregular partisans who fought for the South during the American Civil War.
Cacos - Haitian insurgent groups 19th and 20th centuries.
Camisards – Huguenot insurgency in the beginning of the 18th century in the Cévennes
Cateran - Scottish clan warriors and marauders pre-18th century.
Çetes - Muslim irregulars Asia Minor 1910s-1920s
Cheta - armed bands resisting Ottoman rule in Macedonia, early 20th century.
Chetniks - nationalist movement and guerrilla force in occupied Yugoslavia 1941-44.
Croats (military unit) - 17th century frontier light cavalry in Habsburgh service.
Dubat - indigenous auxiliaries in Italian Somaliand.
Fano - Ethiopian guerrilla force
Fedayeen - Arabic term for fighters willing to sacrifice themselves
Fellagha - nationalist militants in Algeria and Tunisia opposing French colonial rule 1950s.
Filibuster (military) - participants in foreign military interventions without official backing.