Basically a new ListFormWebPart-object must be instanciated, connected with the list and added to the webpartzone of the page. That's it.
Have a look at the following code:
$site = New-Object Microsoft.SharePoint.SPSite -ArgumentList http://sitecollection/sites/site1  
$web = $site.OpenWeb()  
$list = $web.Lists["My Library"] #title of the regarding list  
$webpart = new-object Microsoft.SharePoint.WebPartPages.ListFormWebPart  
# assign list to webpart  
$webpart.ListId = $list.ID  
# possible pagetypes are PAGE_DISPLAYFORM, PAGE_EDITFORM, or PAGE_NEWFORM  
# see: https://msdn.microsoft.com/en-us/library/microsoft.sharepoint.webpartpages.listformwebpart.pagetype.aspx  
$webpart.PageType = "PAGE_EDITFORM"  
# use the absolute link to the form-aspx  
$wpm = $web.GetLimitedWebPartManager("http://sitecollection/sites/site1/myLibrary/Forms/EditForm.aspx", [System.Web.UI.WebControls.WebParts.PersonalizationScope]::Shared)  
$wpm.AddWebPart($webpart, "Main", 1)  
# dispose spweb-object in webpartmanager to avoid memory leaks  
$wpm.Web.Dispose()  
$web.Dispose()  
$site.Dispose()  
